[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:09:14 dist-test-slave-49h3 krb5kdc[8264](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:09:14 dist-test-slave-49h3 krb5kdc[8264](info): set up 2 sockets
Aug 07 05:09:14 dist-test-slave-49h3 krb5kdc[8264](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 05:09:15.082302 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.975s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:09:15 dist-test-slave-49h3 krb5kdc[8264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:09:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:09:15.196708 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:37283
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:37461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:37283
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:15.303051 8277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:15.303303 8277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:15.303342 8277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:15.307287 8277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:15.307344 8277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:15.307361 8277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:15.307376 8277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:15.307389 8277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:15.312564 8277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:37461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:37283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:37283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:15.313570 8277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:15.314559 8277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:15.322882 8283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:15.322897 8285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:15.322901 8282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:15.322935 8277 server_base.cc:1034] running on GCE node
I20260807 05:09:15.323696 8277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:15.324728 8277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:15.325870 8277 hybrid_clock.cc:648] HybridClock initialized: now 1786079355325802 us; error 86 us; skew 500 ppm
Aug 07 05:09:15 dist-test-slave-49h3 krb5kdc[8264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079355, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:15.328462 8277 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:09:15.329372 8277 webserver.cc:460] Webserver started at http://127.8.16.126:44223/ using document root <none> and password file <none>
I20260807 05:09:15.329901 8277 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:15.329962 8277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:15.330116 8277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:15.332150 8277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "a73889b6b574403a881da001a117a136"
format_stamp: "Formatted at 2026-08-07 05:09:15 on dist-test-slave-49h3"
server_key: "1e296668041d2074aafffbc9db56f5ef"
server_key_iv: "05e69c281831bc3b98ee010f5837234b"
server_key_version: "encryptionkey@0"
I20260807 05:09:15.332661 8277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "a73889b6b574403a881da001a117a136"
format_stamp: "Formatted at 2026-08-07 05:09:15 on dist-test-slave-49h3"
server_key: "1e296668041d2074aafffbc9db56f5ef"
server_key_iv: "05e69c281831bc3b98ee010f5837234b"
server_key_version: "encryptionkey@0"
I20260807 05:09:15.336303 8277 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:09:15.338657 8292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:15.339802 8277 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:15.339905 8277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "a73889b6b574403a881da001a117a136"
format_stamp: "Formatted at 2026-08-07 05:09:15 on dist-test-slave-49h3"
server_key: "1e296668041d2074aafffbc9db56f5ef"
server_key_iv: "05e69c281831bc3b98ee010f5837234b"
server_key_version: "encryptionkey@0"
I20260807 05:09:15.340008 8277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:15.368026 8277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:15.371121 8277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:15.371304 8277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:15.377972 8277 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:37283
I20260807 05:09:15.378037 8347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:37283 every 8 connection(s)
I20260807 05:09:15.378896 8277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:15.381549 8348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:15.383271 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 8277
I20260807 05:09:15.383435 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:09:15.385069 8257 external_mini_cluster.cc:1428] Setting key 34034c422e370a5e80d5d1e3f17cdfc5
I20260807 05:09:15.388337 8348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a73889b6b574403a881da001a117a136: Bootstrap starting.
I20260807 05:09:15.390595 8348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a73889b6b574403a881da001a117a136: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:15.391269 8348 log.cc:824] T 00000000000000000000000000000000 P a73889b6b574403a881da001a117a136: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:15.393157 8348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a73889b6b574403a881da001a117a136: No bootstrap required, opened a new log
I20260807 05:09:15.397260 8348 raft_consensus.cc:348] T 00000000000000000000000000000000 P a73889b6b574403a881da001a117a136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73889b6b574403a881da001a117a136" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37283 } }
I20260807 05:09:15.397545 8348 raft_consensus.cc:374] T 00000000000000000000000000000000 P a73889b6b574403a881da001a117a136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:15.397612 8348 raft_consensus.cc:729] T 00000000000000000000000000000000 P a73889b6b574403a881da001a117a136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a73889b6b574403a881da001a117a136, State: Initialized, Role: FOLLOWER
I20260807 05:09:15.398015 8348 consensus_queue.cc:260] T 00000000000000000000000000000000 P a73889b6b574403a881da001a117a136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73889b6b574403a881da001a117a136" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37283 } }
I20260807 05:09:15.398149 8348 raft_consensus.cc:388] T 00000000000000000000000000000000 P a73889b6b574403a881da001a117a136 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:09:15.398191 8348 raft_consensus.cc:482] T 00000000000000000000000000000000 P a73889b6b574403a881da001a117a136 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:09:15.398286 8348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a73889b6b574403a881da001a117a136 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:15.399250 8348 raft_consensus.cc:504] T 00000000000000000000000000000000 P a73889b6b574403a881da001a117a136 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73889b6b574403a881da001a117a136" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37283 } }
I20260807 05:09:15.399606 8348 leader_election.cc:302] T 00000000000000000000000000000000 P a73889b6b574403a881da001a117a136 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a73889b6b574403a881da001a117a136; no voters:
I20260807 05:09:15.399938 8348 leader_election.cc:288] T 00000000000000000000000000000000 P a73889b6b574403a881da001a117a136 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:09:15.400030 8353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a73889b6b574403a881da001a117a136 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:15.400274 8353 raft_consensus.cc:686] T 00000000000000000000000000000000 P a73889b6b574403a881da001a117a136 [term 1 LEADER]: Becoming Leader. State: Replica: a73889b6b574403a881da001a117a136, State: Running, Role: LEADER
I20260807 05:09:15.400596 8353 consensus_queue.cc:237] T 00000000000000000000000000000000 P a73889b6b574403a881da001a117a136 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73889b6b574403a881da001a117a136" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37283 } }
I20260807 05:09:15.401139 8348 sys_catalog.cc:564] T 00000000000000000000000000000000 P a73889b6b574403a881da001a117a136 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:15.401758 8355 sys_catalog.cc:455] T 00000000000000000000000000000000 P a73889b6b574403a881da001a117a136 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a73889b6b574403a881da001a117a136. Latest consensus state: current_term: 1 leader_uuid: "a73889b6b574403a881da001a117a136" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73889b6b574403a881da001a117a136" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37283 } } }
I20260807 05:09:15.401877 8355 sys_catalog.cc:458] T 00000000000000000000000000000000 P a73889b6b574403a881da001a117a136 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:15.402522 8354 sys_catalog.cc:455] T 00000000000000000000000000000000 P a73889b6b574403a881da001a117a136 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a73889b6b574403a881da001a117a136" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73889b6b574403a881da001a117a136" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37283 } } }
I20260807 05:09:15.402621 8354 sys_catalog.cc:458] T 00000000000000000000000000000000 P a73889b6b574403a881da001a117a136 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:15.402890 8359 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 07 05:09:15 dist-test-slave-49h3 krb5kdc[8264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:15.405606 8359 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:15.411001 8359 catalog_manager.cc:1347] Generated new cluster ID: b8146023fdbd4c02ad774a30bd561860
I20260807 05:09:15.411067 8359 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:15.411191 8350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:15.388693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60990 (local address 127.8.16.126:37283)
0807 05:09:15.389204 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:15.389221 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:15.390161 (+ 940us) server_negotiation.cc:407] Connection header received
0807 05:09:15.393314 (+ 3153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:15.393335 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:15.393740 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:15.394096 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:15.396031 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:15.397580 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:15.398915 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:15.399581 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:15.399741 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:15.404635 (+ 4894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:15.404641 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:15.404651 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:15.404678 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:15.406486 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:15.407067 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:15.407073 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:15.407079 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:15.407230 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:15.407616 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:15.407619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:15.407621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:15.408082 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:15.408264 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:15.409879 (+ 1615us) server_negotiation.cc:299] Negotiation successful
0807 05:09:15.410152 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":156,"threads_started":1}
I20260807 05:09:15.427397 8359 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:09:15.428519 8359 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:15.437337 8359 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a73889b6b574403a881da001a117a136: Generated new TSK 0
I20260807 05:09:15.438005 8359 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
I20260807 05:09:15.470796 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37283
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:37461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:15.592337 8374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:15.592592 8374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:15.592638 8374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:15.596629 8374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:15.596691 8374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:15.596758 8374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:09:15.601842 8374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:37461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37283
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:15.602792 8374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:15.603729 8374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:15.612679 8382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:15.612756 8374 server_base.cc:1034] running on GCE node
W20260807 05:09:15.612772 8379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:15.612720 8380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:15.613313 8374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:15.613902 8374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:15.615056 8374 hybrid_clock.cc:648] HybridClock initialized: now 1786079355615032 us; error 33 us; skew 500 ppm
Aug 07 05:09:15 dist-test-slave-49h3 krb5kdc[8264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079355, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:15.617549 8374 init.cc:377] Logged in from keytab as kudu/127.8.16.65@KRBTEST.COM (short username kudu)
I20260807 05:09:15.618377 8374 webserver.cc:460] Webserver started at http://127.8.16.65:38231/ using document root <none> and password file <none>
I20260807 05:09:15.618892 8374 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:15.618957 8374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:15.619148 8374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:15.621090 8374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "5b2cc9e3fe9640e98a31f45fbc0cba7f"
format_stamp: "Formatted at 2026-08-07 05:09:15 on dist-test-slave-49h3"
server_key: "50bb0110c393f62fc95eeb3f9648e97b"
server_key_iv: "5346e8d2020c769381361e0b91895864"
server_key_version: "encryptionkey@0"
I20260807 05:09:15.621671 8374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "5b2cc9e3fe9640e98a31f45fbc0cba7f"
format_stamp: "Formatted at 2026-08-07 05:09:15 on dist-test-slave-49h3"
server_key: "50bb0110c393f62fc95eeb3f9648e97b"
server_key_iv: "5346e8d2020c769381361e0b91895864"
server_key_version: "encryptionkey@0"
I20260807 05:09:15.625253 8374 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 05:09:15.627635 8389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:15.628789 8374 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:15.628893 8374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "5b2cc9e3fe9640e98a31f45fbc0cba7f"
format_stamp: "Formatted at 2026-08-07 05:09:15 on dist-test-slave-49h3"
server_key: "50bb0110c393f62fc95eeb3f9648e97b"
server_key_iv: "5346e8d2020c769381361e0b91895864"
server_key_version: "encryptionkey@0"
I20260807 05:09:15.628983 8374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:15.653057 8374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:15.655897 8374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:15.656085 8374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:15.656684 8374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:15.657531 8374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:15.657580 8374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:15.657631 8374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:15.657666 8374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:15.666630 8374 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:46333
I20260807 05:09:15.666658 8505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:46333 every 8 connection(s)
I20260807 05:09:15.667567 8374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:09:15.667740 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 8374
I20260807 05:09:15.667837 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:09:15.668135 8257 external_mini_cluster.cc:1428] Setting key 7a912b3ae9b9dc05e374c115bc62c351
Aug 07 05:09:15 dist-test-slave-49h3 krb5kdc[8264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079355, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.66@KRBTEST.COM" created.
I20260807 05:09:15.681221 8350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:15.669402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:48949 (local address 127.8.16.126:37283)
0807 05:09:15.669551 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:15.669556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:15.670470 (+ 914us) server_negotiation.cc:407] Connection header received
0807 05:09:15.672068 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:15.672071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:15.672129 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:15.672217 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:15.673317 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:15.674137 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:15.675255 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:15.675608 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:15.675730 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:15.678073 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:15.678076 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:15.678078 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:15.678101 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:15.679223 (+ 1122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:15.679696 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:15.679700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:15.679701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:15.679789 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:15.680083 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:15.680086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:15.680088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:15.680297 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:15.680388 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:15.680924 (+ 536us) server_negotiation.cc:299] Negotiation successful
0807 05:09:15.681057 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 05:09:15.682029 8508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:15.669801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:37283 (local address 127.8.16.65:48949)
0807 05:09:15.670295 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:15.670357 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:15.671864 (+ 1507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:15.672343 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:15.672349 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:15.672705 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:15.673171 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:15.673183 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:15.674259 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:15.674262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:15.675134 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:15.675143 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:15.675748 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:15.675750 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:15.675896 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:15.676448 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:15.676464 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:15.677919 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:15.679325 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:15.679330 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:15.679341 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:15.679587 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:15.679867 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:15.679869 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:15.679870 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:15.679997 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:15.680405 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:15.680411 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:15.680692 (+ 281us) client_negotiation.cc:769] Sending connection context
0807 05:09:15.680848 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 05:09:15.681133 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":110,"threads_started":1}
I20260807 05:09:15.683218 8506 heartbeater.cc:347] Connected to a master server at 127.8.16.126:37283
I20260807 05:09:15.683444 8506 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:15.683915 8506 heartbeater.cc:511] Master 127.8.16.126:37283 requested a full tablet report, sending...
I20260807 05:09:15.685580 8309 ts_manager.cc:194] Registered new tserver with Master: 5b2cc9e3fe9640e98a31f45fbc0cba7f (127.8.16.65:46333)
I20260807 05:09:15.687151 8309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:48949
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
I20260807 05:09:15.694595 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37283
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:37461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:15.801406 8511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:15.801678 8511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:15.801723 8511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:15.805756 8511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:15.805815 8511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:15.805876 8511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:09:15.811007 8511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:37461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37283
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:15.811969 8511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:15.812909 8511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:15.821736 8517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:15.821755 8519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:15.821769 8516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:15.822027 8511 server_base.cc:1034] running on GCE node
I20260807 05:09:15.822402 8511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:15.822942 8511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:15.824108 8511 hybrid_clock.cc:648] HybridClock initialized: now 1786079355824088 us; error 31 us; skew 500 ppm
Aug 07 05:09:15 dist-test-slave-49h3 krb5kdc[8264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079355, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:15.826802 8511 init.cc:377] Logged in from keytab as kudu/127.8.16.66@KRBTEST.COM (short username kudu)
I20260807 05:09:15.827600 8511 webserver.cc:460] Webserver started at http://127.8.16.66:39727/ using document root <none> and password file <none>
I20260807 05:09:15.828112 8511 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:15.828176 8511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:15.828361 8511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:15.830396 8511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "61c67e5f4c9c4eb6b24eb5efb336467a"
format_stamp: "Formatted at 2026-08-07 05:09:15 on dist-test-slave-49h3"
server_key: "02f966c7d563b38586dfa62724cd82a7"
server_key_iv: "3ed9d4892a5e72fae0a8c4890b6d9f12"
server_key_version: "encryptionkey@0"
I20260807 05:09:15.830924 8511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "61c67e5f4c9c4eb6b24eb5efb336467a"
format_stamp: "Formatted at 2026-08-07 05:09:15 on dist-test-slave-49h3"
server_key: "02f966c7d563b38586dfa62724cd82a7"
server_key_iv: "3ed9d4892a5e72fae0a8c4890b6d9f12"
server_key_version: "encryptionkey@0"
I20260807 05:09:15.834610 8511 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:09:15.836961 8526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:15.838187 8511 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:15.838316 8511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "61c67e5f4c9c4eb6b24eb5efb336467a"
format_stamp: "Formatted at 2026-08-07 05:09:15 on dist-test-slave-49h3"
server_key: "02f966c7d563b38586dfa62724cd82a7"
server_key_iv: "3ed9d4892a5e72fae0a8c4890b6d9f12"
server_key_version: "encryptionkey@0"
I20260807 05:09:15.838414 8511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:15.851997 8511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:15.854717 8511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:15.854867 8511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:15.855410 8511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:15.856174 8511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:15.856220 8511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:15.856256 8511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:15.856277 8511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:15.864754 8511 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:45227
I20260807 05:09:15.864782 8641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:45227 every 8 connection(s)
I20260807 05:09:15.865640 8511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:09:15.871011 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 8511
I20260807 05:09:15.871135 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:09:15.871399 8257 external_mini_cluster.cc:1428] Setting key 28d34cedff4999afacf58c0d0ee7a88d
Aug 07 05:09:15 dist-test-slave-49h3 krb5kdc[8264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079355, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:15.879196 8350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:15.867335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:40329 (local address 127.8.16.126:37283)
0807 05:09:15.867463 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:15.867467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:15.868238 (+ 771us) server_negotiation.cc:407] Connection header received
0807 05:09:15.869632 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:15.869634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:15.869680 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:15.869750 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:15.870866 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:15.871618 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:15.873103 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:15.873396 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:15.873533 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:15.875997 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:15.876000 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:15.876002 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:15.876025 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:15.877259 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:15.877773 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:15.877776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:15.877777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:15.877850 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:15.878153 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:15.878157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:15.878159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:15.878332 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:15.878418 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:15.878901 (+ 483us) server_negotiation.cc:299] Negotiation successful
0807 05:09:15.879023 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:09:15.879982 8645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:15.867660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:37283 (local address 127.8.16.66:40329)
0807 05:09:15.868086 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:15.868129 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:15.869351 (+ 1222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:15.869896 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:15.869902 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:15.870253 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:15.870723 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:15.870735 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:15.871736 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:15.871738 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:15.872981 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:15.872991 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:15.873566 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:15.873570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:15.873709 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:15.874270 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:15.874285 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:15.875849 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:15.877366 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:15.877372 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:15.877387 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:15.877681 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:15.877929 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:15.877932 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:15.877934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:15.878064 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:15.878428 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:15.878435 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:15.878682 (+ 247us) client_negotiation.cc:769] Sending connection context
0807 05:09:15.878847 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 05:09:15.879160 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":109,"threads_started":1}
WARNING: no policy specified for kudu/127.8.16.67@KRBTEST.COM; defaulting to no policy
I20260807 05:09:15.881091 8643 heartbeater.cc:347] Connected to a master server at 127.8.16.126:37283
I20260807 05:09:15.881325 8643 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:15.881855 8643 heartbeater.cc:511] Master 127.8.16.126:37283 requested a full tablet report, sending...
I20260807 05:09:15.882850 8309 ts_manager.cc:194] Registered new tserver with Master: 61c67e5f4c9c4eb6b24eb5efb336467a (127.8.16.66:45227)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.67@KRBTEST.COM" created.
I20260807 05:09:15.883525 8309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:40329
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
I20260807 05:09:15.897570 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37283
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:37461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:16.008551 8648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:16.008816 8648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:16.008860 8648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:16.012859 8648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:16.012938 8648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:16.013003 8648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:09:16.018257 8648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:37461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37283
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:16.019263 8648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:16.020236 8648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:16.029464 8654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:16.029477 8656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:16.029477 8653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:16.029789 8648 server_base.cc:1034] running on GCE node
I20260807 05:09:16.030369 8648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:16.030956 8648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:16.032100 8648 hybrid_clock.cc:648] HybridClock initialized: now 1786079356032086 us; error 24 us; skew 500 ppm
Aug 07 05:09:16 dist-test-slave-49h3 krb5kdc[8264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079356, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:16.034897 8648 init.cc:377] Logged in from keytab as kudu/127.8.16.67@KRBTEST.COM (short username kudu)
I20260807 05:09:16.035737 8648 webserver.cc:460] Webserver started at http://127.8.16.67:38415/ using document root <none> and password file <none>
I20260807 05:09:16.036242 8648 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:16.036296 8648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:16.036461 8648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:16.038460 8648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "72f7ce83d5a1494bbb6fc68291348c10"
format_stamp: "Formatted at 2026-08-07 05:09:16 on dist-test-slave-49h3"
server_key: "aa59d118981b5e420af54620f9399438"
server_key_iv: "d710582e1f817edc1875be7be9cc3774"
server_key_version: "encryptionkey@0"
I20260807 05:09:16.038961 8648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "72f7ce83d5a1494bbb6fc68291348c10"
format_stamp: "Formatted at 2026-08-07 05:09:16 on dist-test-slave-49h3"
server_key: "aa59d118981b5e420af54620f9399438"
server_key_iv: "d710582e1f817edc1875be7be9cc3774"
server_key_version: "encryptionkey@0"
I20260807 05:09:16.042433 8648 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 05:09:16.044982 8663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:16.046160 8648 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:09:16.046254 8648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "72f7ce83d5a1494bbb6fc68291348c10"
format_stamp: "Formatted at 2026-08-07 05:09:16 on dist-test-slave-49h3"
server_key: "aa59d118981b5e420af54620f9399438"
server_key_iv: "d710582e1f817edc1875be7be9cc3774"
server_key_version: "encryptionkey@0"
I20260807 05:09:16.046339 8648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:16.067004 8648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:16.069682 8648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:16.069834 8648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:16.070365 8648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:16.071143 8648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:16.071192 8648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:16.071229 8648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:16.071249 8648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:16.079916 8648 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:38095
I20260807 05:09:16.079964 8779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:38095 every 8 connection(s)
I20260807 05:09:16.080775 8648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:09:16.083978 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 8648
I20260807 05:09:16.084079 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:09:16.084314 8257 external_mini_cluster.cc:1428] Setting key 8073fb32b231746820df6c0ad313be12
Aug 07 05:09:16 dist-test-slave-49h3 krb5kdc[8264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079356, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:16.093613 8350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:16.082491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:35889 (local address 127.8.16.126:37283)
0807 05:09:16.082636 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:16.082641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:16.083483 (+ 842us) server_negotiation.cc:407] Connection header received
0807 05:09:16.084796 (+ 1313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:16.084799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:16.084847 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:16.084935 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:16.086255 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:16.086782 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:16.087942 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:16.088156 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:16.088242 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:16.090575 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:16.090578 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:16.090579 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:16.090602 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:16.091719 (+ 1117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:16.092183 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:16.092187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:16.092188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:16.092266 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:16.092564 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:16.092567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:16.092569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:16.092739 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:16.092833 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:16.093301 (+ 468us) server_negotiation.cc:299] Negotiation successful
0807 05:09:16.093426 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 05:09:16.094367 8782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:16.082881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:37283 (local address 127.8.16.67:35889)
0807 05:09:16.083330 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:16.083371 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:16.084578 (+ 1207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:16.085112 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:16.085119 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:16.085592 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:16.086108 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:16.086123 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:16.086903 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:16.086905 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:16.087818 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:16.087828 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:16.088297 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:16.088301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:16.088462 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:16.089027 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:16.089044 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:16.090418 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:16.091832 (+ 1414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:16.091837 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:16.091851 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:16.092090 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:16.092345 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:16.092349 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:16.092351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:16.092478 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:16.092843 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:16.092850 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:16.093079 (+ 229us) client_negotiation.cc:769] Sending connection context
0807 05:09:16.093239 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 05:09:16.093586 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":117,"threads_started":1}
I20260807 05:09:16.095376 8780 heartbeater.cc:347] Connected to a master server at 127.8.16.126:37283
I20260807 05:09:16.095592 8780 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:16.096047 8780 heartbeater.cc:511] Master 127.8.16.126:37283 requested a full tablet report, sending...
I20260807 05:09:16.097023 8309 ts_manager.cc:194] Registered new tserver with Master: 72f7ce83d5a1494bbb6fc68291348c10 (127.8.16.67:38095)
I20260807 05:09:16.097702 8309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:35889
I20260807 05:09:16.098372 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 05:09:16 dist-test-slave-49h3 krb5kdc[8264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079356, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 05:09:16 dist-test-slave-49h3 krb5kdc[8264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079356, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.16.65@KRBTEST.COM
I20260807 05:09:16.123833 8787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:16.115070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46706 (local address 127.8.16.65:46333)
0807 05:09:16.115411 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:16.115421 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:16.115444 (+ 23us) server_negotiation.cc:407] Connection header received
0807 05:09:16.115493 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:16.115498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:16.115686 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:16.115806 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:16.116458 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:16.117249 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:16.118098 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:16.118465 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:16.118561 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:16.120758 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:16.120767 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:16.120778 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:16.120801 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:16.122206 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:16.122591 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:16.122596 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:16.122597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:16.122674 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:16.122909 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:16.122912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:16.122914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:16.123139 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:16.123358 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:16.123515 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 05:09:16.123660 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":116,"threads_started":1}
W20260807 05:09:16.124749 8417 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46706
Aug 07 05:09:16 dist-test-slave-49h3 krb5kdc[8264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 05:09:16 dist-test-slave-49h3 krb5kdc[8264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.65@KRBTEST.COM
I20260807 05:09:16.150482 8787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:16.142224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46716 (local address 127.8.16.65:46333)
0807 05:09:16.142349 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:16.142353 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:16.142517 (+ 164us) server_negotiation.cc:407] Connection header received
0807 05:09:16.142661 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:16.142664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:16.142714 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:16.142776 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:16.143532 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:16.144059 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:16.144879 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:16.145154 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:16.145249 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:16.147550 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:16.147554 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:16.147556 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:16.147582 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:16.148823 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:16.149199 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:16.149201 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:16.149202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:16.149269 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:16.149567 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:16.149571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:16.149573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:16.149755 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:16.149914 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:16.150123 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 05:09:16.150244 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 05:09:16.151510 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 8374
I20260807 05:09:16.158061 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 8511
I20260807 05:09:16.163476 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 8648
I20260807 05:09:16.171116 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 8277
2026-08-07T05:09:16Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2093 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:09:16 dist-test-slave-49h3 krb5kdc[8794](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:09:16 dist-test-slave-49h3 krb5kdc[8794](info): set up 2 sockets
Aug 07 05:09:16 dist-test-slave-49h3 krb5kdc[8794](info): commencing operation
krb5kdc: starting...
W20260807 05:09:17.148963 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:09:17 dist-test-slave-49h3 krb5kdc[8794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:09:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:09:17.264732 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:36453
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:45675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:36453
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:17.370775 8807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:17.371042 8807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:17.371080 8807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:17.375005 8807 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:17.375070 8807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:17.375088 8807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:17.375103 8807 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:17.375118 8807 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:17.380581 8807 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:45675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:36453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:36453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:17.381628 8807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:17.382584 8807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:17.390693 8815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:17.390693 8813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:17.390689 8812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:17.391064 8807 server_base.cc:1034] running on GCE node
I20260807 05:09:17.391628 8807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:17.392705 8807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:17.393869 8807 hybrid_clock.cc:648] HybridClock initialized: now 1786079357393850 us; error 36 us; skew 500 ppm
Aug 07 05:09:17 dist-test-slave-49h3 krb5kdc[8794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079357, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:17.396607 8807 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:09:17.397487 8807 webserver.cc:460] Webserver started at http://127.8.16.126:35737/ using document root <none> and password file <none>
I20260807 05:09:17.398021 8807 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:17.398100 8807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:17.398336 8807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:17.400378 8807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "63b0ea5124f445b894da7b8565133fee"
format_stamp: "Formatted at 2026-08-07 05:09:17 on dist-test-slave-49h3"
server_key: "c662aa8eb697858dfa33470fe5903bed"
server_key_iv: "c87c724f0e0217ea9c310fb9db0d6f40"
server_key_version: "encryptionkey@0"
I20260807 05:09:17.400923 8807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "63b0ea5124f445b894da7b8565133fee"
format_stamp: "Formatted at 2026-08-07 05:09:17 on dist-test-slave-49h3"
server_key: "c662aa8eb697858dfa33470fe5903bed"
server_key_iv: "c87c724f0e0217ea9c310fb9db0d6f40"
server_key_version: "encryptionkey@0"
I20260807 05:09:17.404649 8807 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:09:17.407045 8822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:17.408253 8807 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:17.408367 8807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "63b0ea5124f445b894da7b8565133fee"
format_stamp: "Formatted at 2026-08-07 05:09:17 on dist-test-slave-49h3"
server_key: "c662aa8eb697858dfa33470fe5903bed"
server_key_iv: "c87c724f0e0217ea9c310fb9db0d6f40"
server_key_version: "encryptionkey@0"
I20260807 05:09:17.408465 8807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:17.421375 8807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:17.424408 8807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:17.424587 8807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:17.431144 8807 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:36453
I20260807 05:09:17.431196 8876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:36453 every 8 connection(s)
I20260807 05:09:17.432049 8807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:17.434692 8877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:17.439877 8877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63b0ea5124f445b894da7b8565133fee: Bootstrap starting.
I20260807 05:09:17.440698 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 8807
I20260807 05:09:17.440802 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:09:17.441022 8257 external_mini_cluster.cc:1428] Setting key ec4880a49cbdafa7d0196d25cfba11c7
I20260807 05:09:17.442430 8877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63b0ea5124f445b894da7b8565133fee: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:17.443125 8877 log.cc:824] T 00000000000000000000000000000000 P 63b0ea5124f445b894da7b8565133fee: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:17.445207 8877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63b0ea5124f445b894da7b8565133fee: No bootstrap required, opened a new log
I20260807 05:09:17.449388 8877 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63b0ea5124f445b894da7b8565133fee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b0ea5124f445b894da7b8565133fee" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36453 } }
I20260807 05:09:17.449642 8877 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63b0ea5124f445b894da7b8565133fee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:17.449709 8877 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63b0ea5124f445b894da7b8565133fee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63b0ea5124f445b894da7b8565133fee, State: Initialized, Role: FOLLOWER
Aug 07 05:09:17 dist-test-slave-49h3 krb5kdc[8794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:17.450129 8877 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63b0ea5124f445b894da7b8565133fee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b0ea5124f445b894da7b8565133fee" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36453 } }
I20260807 05:09:17.450263 8877 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63b0ea5124f445b894da7b8565133fee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:09:17.450326 8877 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63b0ea5124f445b894da7b8565133fee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:09:17.450423 8877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63b0ea5124f445b894da7b8565133fee [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:17.451555 8877 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63b0ea5124f445b894da7b8565133fee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b0ea5124f445b894da7b8565133fee" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36453 } }
I20260807 05:09:17.451925 8877 leader_election.cc:302] T 00000000000000000000000000000000 P 63b0ea5124f445b894da7b8565133fee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63b0ea5124f445b894da7b8565133fee; no voters:
I20260807 05:09:17.452248 8877 leader_election.cc:288] T 00000000000000000000000000000000 P 63b0ea5124f445b894da7b8565133fee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:09:17.452339 8882 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63b0ea5124f445b894da7b8565133fee [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:17.452553 8882 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63b0ea5124f445b894da7b8565133fee [term 1 LEADER]: Becoming Leader. State: Replica: 63b0ea5124f445b894da7b8565133fee, State: Running, Role: LEADER
I20260807 05:09:17.452845 8882 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63b0ea5124f445b894da7b8565133fee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b0ea5124f445b894da7b8565133fee" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36453 } }
I20260807 05:09:17.453565 8877 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63b0ea5124f445b894da7b8565133fee [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:17.454046 8884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63b0ea5124f445b894da7b8565133fee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63b0ea5124f445b894da7b8565133fee. Latest consensus state: current_term: 1 leader_uuid: "63b0ea5124f445b894da7b8565133fee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b0ea5124f445b894da7b8565133fee" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36453 } } }
I20260807 05:09:17.454181 8884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63b0ea5124f445b894da7b8565133fee [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:17.454484 8883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63b0ea5124f445b894da7b8565133fee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63b0ea5124f445b894da7b8565133fee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b0ea5124f445b894da7b8565133fee" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36453 } } }
I20260807 05:09:17.454568 8887 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:17.454576 8883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63b0ea5124f445b894da7b8565133fee [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:17.456265 8880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:17.442075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56936 (local address 127.8.16.126:36453)
0807 05:09:17.442463 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:17.442478 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:17.442513 (+ 35us) server_negotiation.cc:407] Connection header received
0807 05:09:17.443148 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:17.443165 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:17.443532 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:17.443887 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:17.444713 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:17.445789 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:17.446983 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:17.447475 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:17.447600 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:17.450451 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:17.450458 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:17.450474 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:17.450508 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:17.452429 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:17.452920 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:17.452925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:17.452931 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:17.453063 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:17.453369 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:17.453373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:17.453375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:17.453904 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:17.454103 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:17.454953 (+ 850us) server_negotiation.cc:299] Negotiation successful
0807 05:09:17.455286 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":108,"threads_started":1}
I20260807 05:09:17.457633 8887 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:17.462836 8887 catalog_manager.cc:1347] Generated new cluster ID: 452c69c1b277459387492b1e2bf79fc2
I20260807 05:09:17.462890 8887 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:17.475716 8887 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:09:17.476986 8887 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:17.483695 8887 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63b0ea5124f445b894da7b8565133fee: Generated new TSK 0
I20260807 05:09:17.484462 8887 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
I20260807 05:09:17.512164 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36453
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:45675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:17.620518 8903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:17.620774 8903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:17.620816 8903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:17.624775 8903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:17.624831 8903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:17.624897 8903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:09:17.630169 8903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:45675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36453
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:17.631176 8903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:17.632164 8903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:17.641130 8908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:17.641216 8909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:17.641203 8903 server_base.cc:1034] running on GCE node
W20260807 05:09:17.641137 8911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:17.641775 8903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:17.642375 8903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:17.643539 8903 hybrid_clock.cc:648] HybridClock initialized: now 1786079357643522 us; error 29 us; skew 500 ppm
Aug 07 05:09:17 dist-test-slave-49h3 krb5kdc[8794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079357, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:17.646153 8903 init.cc:377] Logged in from keytab as kudu/127.8.16.65@KRBTEST.COM (short username kudu)
I20260807 05:09:17.646975 8903 webserver.cc:460] Webserver started at http://127.8.16.65:43321/ using document root <none> and password file <none>
I20260807 05:09:17.647497 8903 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:17.647562 8903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:17.647770 8903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:17.649770 8903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "47d5246a1357428eb793c0b73dddedab"
format_stamp: "Formatted at 2026-08-07 05:09:17 on dist-test-slave-49h3"
server_key: "3c5b11e34124bc3236ee3f3588c4aa86"
server_key_iv: "9005e7d2d4bf54a60ae7eab45379c456"
server_key_version: "encryptionkey@0"
I20260807 05:09:17.650312 8903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "47d5246a1357428eb793c0b73dddedab"
format_stamp: "Formatted at 2026-08-07 05:09:17 on dist-test-slave-49h3"
server_key: "3c5b11e34124bc3236ee3f3588c4aa86"
server_key_iv: "9005e7d2d4bf54a60ae7eab45379c456"
server_key_version: "encryptionkey@0"
I20260807 05:09:17.653905 8903 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 05:09:17.656315 8918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:17.657490 8903 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:17.657608 8903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "47d5246a1357428eb793c0b73dddedab"
format_stamp: "Formatted at 2026-08-07 05:09:17 on dist-test-slave-49h3"
server_key: "3c5b11e34124bc3236ee3f3588c4aa86"
server_key_iv: "9005e7d2d4bf54a60ae7eab45379c456"
server_key_version: "encryptionkey@0"
I20260807 05:09:17.657702 8903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:17.670480 8903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:17.673419 8903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:17.673616 8903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:17.674162 8903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:17.674932 8903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:17.674978 8903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:17.675014 8903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:17.675035 8903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:17.683326 8903 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:37625
I20260807 05:09:17.683377 9033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:37625 every 8 connection(s)
I20260807 05:09:17.684197 8903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:09:17.688452 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 8903
I20260807 05:09:17.688547 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:09:17.688787 8257 external_mini_cluster.cc:1428] Setting key 16713bc96b0e96181cc4151fa2ee80ac
Aug 07 05:09:17 dist-test-slave-49h3 krb5kdc[8794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079357, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.16.66@KRBTEST.COM; defaulting to no policy
I20260807 05:09:17.698024 8880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:17.686006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:40769 (local address 127.8.16.126:36453)
0807 05:09:17.686144 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:17.686148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:17.686928 (+ 780us) server_negotiation.cc:407] Connection header received
0807 05:09:17.688229 (+ 1301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:17.688233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:17.688290 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:17.688362 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:17.689726 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:17.690370 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:17.691880 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:17.692173 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:17.692268 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:17.694674 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:17.694677 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:17.694679 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:17.694702 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:17.696013 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:17.696487 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:17.696490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:17.696491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:17.696568 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:17.696884 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:17.696888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:17.696890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:17.697080 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:17.697173 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:17.697718 (+ 545us) server_negotiation.cc:299] Negotiation successful
0807 05:09:17.697852 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 05:09:17.698868 9036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:17.686346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36453 (local address 127.8.16.65:40769)
0807 05:09:17.686779 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:17.686821 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:17.688022 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:17.688538 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:17.688547 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:17.688938 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:17.689580 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:17.689598 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:17.690502 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:17.690506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:17.691754 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:17.691766 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:17.692323 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:17.692328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:17.692498 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:17.693063 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:17.693087 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:17.694517 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:17.696128 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:17.696133 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:17.696150 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:17.696393 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:17.696660 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:17.696663 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:17.696665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:17.696799 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:17.697185 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:17.697192 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:17.697441 (+ 249us) client_negotiation.cc:769] Sending connection context
0807 05:09:17.697664 (+ 223us) client_negotiation.cc:240] Negotiation successful
0807 05:09:17.697988 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":117,"threads_started":1}
I20260807 05:09:17.700139 9034 heartbeater.cc:347] Connected to a master server at 127.8.16.126:36453
I20260807 05:09:17.700381 9034 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.66@KRBTEST.COM" created.
I20260807 05:09:17.700937 9034 heartbeater.cc:511] Master 127.8.16.126:36453 requested a full tablet report, sending...
I20260807 05:09:17.702641 8839 ts_manager.cc:194] Registered new tserver with Master: 47d5246a1357428eb793c0b73dddedab (127.8.16.65:37625)
I20260807 05:09:17.703783 8839 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:40769
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
I20260807 05:09:17.716140 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36453
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:45675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:17.823362 9039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:17.823622 9039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:17.823683 9039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:17.827695 9039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:17.827755 9039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:17.827836 9039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:09:17.832899 9039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:45675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36453
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:17.833889 9039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:17.834827 9039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:17.843839 9045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:17.843883 9039 server_base.cc:1034] running on GCE node
W20260807 05:09:17.843833 9044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:17.843833 9047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:17.844488 9039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:17.845072 9039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:17.846228 9039 hybrid_clock.cc:648] HybridClock initialized: now 1786079357846202 us; error 36 us; skew 500 ppm
Aug 07 05:09:17 dist-test-slave-49h3 krb5kdc[8794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079357, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:17.848795 9039 init.cc:377] Logged in from keytab as kudu/127.8.16.66@KRBTEST.COM (short username kudu)
I20260807 05:09:17.849642 9039 webserver.cc:460] Webserver started at http://127.8.16.66:33555/ using document root <none> and password file <none>
I20260807 05:09:17.850142 9039 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:17.850198 9039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:17.850364 9039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:17.852368 9039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "dd163eb175a14f70919ceaff91fd797b"
format_stamp: "Formatted at 2026-08-07 05:09:17 on dist-test-slave-49h3"
server_key: "3428141c933610470a2b62339e9ebef1"
server_key_iv: "b9d51e323cd4f1fc6d4fe697a9cfb1c2"
server_key_version: "encryptionkey@0"
I20260807 05:09:17.852878 9039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "dd163eb175a14f70919ceaff91fd797b"
format_stamp: "Formatted at 2026-08-07 05:09:17 on dist-test-slave-49h3"
server_key: "3428141c933610470a2b62339e9ebef1"
server_key_iv: "b9d51e323cd4f1fc6d4fe697a9cfb1c2"
server_key_version: "encryptionkey@0"
I20260807 05:09:17.856662 9039 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 05:09:17.858951 9054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:17.860074 9039 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:17.860203 9039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "dd163eb175a14f70919ceaff91fd797b"
format_stamp: "Formatted at 2026-08-07 05:09:17 on dist-test-slave-49h3"
server_key: "3428141c933610470a2b62339e9ebef1"
server_key_iv: "b9d51e323cd4f1fc6d4fe697a9cfb1c2"
server_key_version: "encryptionkey@0"
I20260807 05:09:17.860296 9039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:17.890870 9039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:17.893846 9039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:17.894034 9039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:17.894623 9039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:17.895430 9039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:17.895479 9039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:17.895531 9039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:17.895560 9039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:17.903896 9039 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:37305
I20260807 05:09:17.903926 9169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:37305 every 8 connection(s)
I20260807 05:09:17.904803 9039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:09:17.912822 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 9039
I20260807 05:09:17.912954 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:09:17.913291 8257 external_mini_cluster.cc:1428] Setting key 1e023e36b91c3a6d20014819b4b494db
Aug 07 05:09:17 dist-test-slave-49h3 krb5kdc[8794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079357, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:17.917868 8880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:17.906641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:57731 (local address 127.8.16.126:36453)
0807 05:09:17.906784 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:17.906788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:17.907568 (+ 780us) server_negotiation.cc:407] Connection header received
0807 05:09:17.908821 (+ 1253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:17.908824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:17.908870 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:17.908941 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:17.910074 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:17.910595 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:17.911680 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:17.911905 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:17.911974 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:17.914631 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:17.914635 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:17.914638 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:17.914665 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:17.915931 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:17.916465 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:17.916468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:17.916470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:17.916560 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:17.916862 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:17.916865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:17.916866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:17.917067 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:17.917178 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:17.917617 (+ 439us) server_negotiation.cc:299] Negotiation successful
0807 05:09:17.917734 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 05:09:17.918808 9172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:17.906969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36453 (local address 127.8.16.66:57731)
0807 05:09:17.907422 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:17.907464 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:17.908644 (+ 1180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:17.909092 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:17.909099 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:17.909439 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:17.909949 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:17.909963 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:17.910682 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:17.910684 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:17.911580 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:17.911589 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:17.912074 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:17.912077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:17.912221 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:17.912766 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:17.912787 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:17.914473 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:17.916039 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:17.916045 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:17.916059 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:17.916377 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:17.916645 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:17.916648 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:17.916650 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:17.916784 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:17.917159 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:17.917163 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:17.917379 (+ 216us) client_negotiation.cc:769] Sending connection context
0807 05:09:17.917574 (+ 195us) client_negotiation.cc:240] Negotiation successful
0807 05:09:17.917848 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":121,"threads_started":1}
I20260807 05:09:17.919842 9170 heartbeater.cc:347] Connected to a master server at 127.8.16.126:36453
I20260807 05:09:17.920056 9170 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:17.920512 9170 heartbeater.cc:511] Master 127.8.16.126:36453 requested a full tablet report, sending...
I20260807 05:09:17.921427 8839 ts_manager.cc:194] Registered new tserver with Master: dd163eb175a14f70919ceaff91fd797b (127.8.16.66:37305)
I20260807 05:09:17.921993 8839 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:57731
WARNING: no policy specified for kudu/127.8.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
I20260807 05:09:17.941295 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36453
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:45675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:18.050710 9175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:18.050974 9175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:18.051020 9175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:18.055099 9175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:18.055162 9175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:18.055233 9175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:09:18.060561 9175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:45675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36453
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:18.061676 9175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:18.062672 9175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:18.072149 9180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:18.072181 9183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:18.072479 9181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:18.072649 9175 server_base.cc:1034] running on GCE node
I20260807 05:09:18.073038 9175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:18.073689 9175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:18.074851 9175 hybrid_clock.cc:648] HybridClock initialized: now 1786079358074834 us; error 28 us; skew 500 ppm
Aug 07 05:09:18 dist-test-slave-49h3 krb5kdc[8794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079358, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:18.077593 9175 init.cc:377] Logged in from keytab as kudu/127.8.16.67@KRBTEST.COM (short username kudu)
I20260807 05:09:18.078394 9175 webserver.cc:460] Webserver started at http://127.8.16.67:41275/ using document root <none> and password file <none>
I20260807 05:09:18.078924 9175 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:18.078985 9175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:18.079175 9175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:18.081163 9175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "d7f7a43d89534957a1732dafed1e17e1"
format_stamp: "Formatted at 2026-08-07 05:09:18 on dist-test-slave-49h3"
server_key: "ba9af501b3882146f32e503afaf9a49e"
server_key_iv: "552b0f2c0da327824ead71201543f8b3"
server_key_version: "encryptionkey@0"
I20260807 05:09:18.081719 9175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "d7f7a43d89534957a1732dafed1e17e1"
format_stamp: "Formatted at 2026-08-07 05:09:18 on dist-test-slave-49h3"
server_key: "ba9af501b3882146f32e503afaf9a49e"
server_key_iv: "552b0f2c0da327824ead71201543f8b3"
server_key_version: "encryptionkey@0"
I20260807 05:09:18.085292 9175 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 05:09:18.087598 9190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:18.088798 9175 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:18.088913 9175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "d7f7a43d89534957a1732dafed1e17e1"
format_stamp: "Formatted at 2026-08-07 05:09:18 on dist-test-slave-49h3"
server_key: "ba9af501b3882146f32e503afaf9a49e"
server_key_iv: "552b0f2c0da327824ead71201543f8b3"
server_key_version: "encryptionkey@0"
I20260807 05:09:18.089005 9175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:18.104502 9175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:18.107824 9175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:18.108009 9175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:18.108610 9175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:18.109436 9175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:18.109519 9175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:18.109586 9175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:18.109611 9175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:18.118069 9175 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:32845
I20260807 05:09:18.118121 9305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:32845 every 8 connection(s)
I20260807 05:09:18.118939 9175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:09:18.127852 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 9175
I20260807 05:09:18.127969 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
Aug 07 05:09:18 dist-test-slave-49h3 krb5kdc[8794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079358, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:18.128296 8257 external_mini_cluster.cc:1428] Setting key 90b0df2b99a20b6cd9047a10d0d38eb4
I20260807 05:09:18.132241 8880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:18.120720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:60041 (local address 127.8.16.126:36453)
0807 05:09:18.120845 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:18.120849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:18.121702 (+ 853us) server_negotiation.cc:407] Connection header received
0807 05:09:18.122943 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:18.122945 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:18.122991 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:18.123063 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:18.124276 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.124793 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.126001 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.126230 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.126309 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:18.128742 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:18.128746 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:18.128749 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:18.128777 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:18.130284 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:18.130785 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:18.130788 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:18.130789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:18.130869 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:18.131246 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:18.131249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:18.131250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:18.131441 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:18.131518 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:18.131974 (+ 456us) server_negotiation.cc:299] Negotiation successful
0807 05:09:18.132115 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 05:09:18.133033 9308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:18.121090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36453 (local address 127.8.16.67:60041)
0807 05:09:18.121551 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:18.121601 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:18.122766 (+ 1165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:18.123312 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:18.123319 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:18.123663 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:18.124145 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:18.124158 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.124911 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.124913 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:18.125891 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:18.125901 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.126353 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.126356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:18.126495 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:18.127019 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:18.127036 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:18.128574 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:18.130384 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:18.130390 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:18.130406 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:18.130706 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:18.130981 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:18.130984 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:18.130986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:18.131138 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:18.131523 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:18.131528 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:18.131777 (+ 249us) client_negotiation.cc:769] Sending connection context
0807 05:09:18.131934 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 05:09:18.132199 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":115,"threads_started":1}
I20260807 05:09:18.134127 9306 heartbeater.cc:347] Connected to a master server at 127.8.16.126:36453
I20260807 05:09:18.134357 9306 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:18.134825 9306 heartbeater.cc:511] Master 127.8.16.126:36453 requested a full tablet report, sending...
I20260807 05:09:18.135851 8839 ts_manager.cc:194] Registered new tserver with Master: d7f7a43d89534957a1732dafed1e17e1 (127.8.16.67:32845)
I20260807 05:09:18.136389 8839 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:60041
I20260807 05:09:18.142256 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:09:18.151842 8880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:18.145528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56950 (local address 127.8.16.126:36453)
0807 05:09:18.145659 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:18.145664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:18.145831 (+ 167us) server_negotiation.cc:407] Connection header received
0807 05:09:18.145935 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:18.145937 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:18.145977 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:18.146040 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:18.146694 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.147199 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.147971 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.148179 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.148239 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:18.149415 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:18.149417 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:18.149419 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:18.149435 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:18.150581 (+ 1146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:18.150916 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:18.150918 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:18.150920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:18.150983 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:18.151178 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:18.151180 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:18.151181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:18.151370 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:18.151466 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:18.151609 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 05:09:18.151695 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 05:09:18.156476 8839 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:09:18.159394 8839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:09:18.173576 9318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:18.167859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:32845 (local address 127.0.0.1:46590)
0807 05:09:18.168209 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:18.168262 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:18.168397 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:18.169151 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:18.169157 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:18.169188 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:18.169355 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:18.169362 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.170773 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.170776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:18.172099 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:18.172108 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.173078 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.173082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:18.173186 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:18.173204 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 05:09:18.173254 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 05:09:18.173390 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":50,"spinlock_wait_cycles":11392,"thread_start_us":88,"threads_started":1}
I20260807 05:09:18.173684 9319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:18.167734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36476 (local address 127.8.16.65:37625)
0807 05:09:18.168675 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:18.168689 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:18.168709 (+ 20us) server_negotiation.cc:407] Connection header received
0807 05:09:18.168787 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:18.168791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:18.168980 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:18.169713 (+ 733us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:18.169863 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.170874 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.172049 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.172870 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.172974 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:18.173125 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:18.173388 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 05:09:18.173525 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":829,"thread_start_us":95,"threads_started":1}
I20260807 05:09:18.173691 9323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:18.168690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43286 (local address 127.8.16.66:37305)
0807 05:09:18.168966 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:18.168977 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:18.168998 (+ 21us) server_negotiation.cc:407] Connection header received
0807 05:09:18.169071 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:18.169076 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:18.169259 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:18.169697 (+ 438us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:18.169821 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.170534 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.172412 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.173154 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.173260 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:18.173390 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:18.173468 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 05:09:18.173542 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":86,"threads_started":1}
I20260807 05:09:18.174052 9320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:18.167574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:37625 (local address 127.0.0.1:36476)
0807 05:09:18.168209 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:18.168262 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:18.168396 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:18.169151 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:18.169157 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:18.169188 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:18.169565 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:18.169572 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.170988 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.170991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:18.171942 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:18.171951 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.173004 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.173008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:18.173119 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:18.173171 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 05:09:18.173268 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 05:09:18.173930 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":636,"spinlock_wait_cycles":5888,"thread_start_us":98,"threads_started":1}
I20260807 05:09:18.174165 9322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:18.168283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:37305 (local address 127.0.0.1:43286)
0807 05:09:18.168512 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:18.168525 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:18.168615 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:18.169410 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:18.169413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:18.169428 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:18.169550 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:18.169556 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.170650 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.170654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:18.172306 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:18.172313 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.173265 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.173268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:18.173352 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:18.173371 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 05:09:18.173423 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 05:09:18.173522 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":237,"thread_start_us":85,"threads_started":1}
I20260807 05:09:18.174587 9321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:18.168050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46590 (local address 127.8.16.67:32845)
0807 05:09:18.168674 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:18.168687 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:18.168709 (+ 22us) server_negotiation.cc:407] Connection header received
0807 05:09:18.168780 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:18.168785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:18.168980 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:18.169260 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:18.169485 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.170649 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.172209 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.172957 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.174233 (+ 1276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:18.174364 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:18.174409 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 05:09:18.174478 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":141,"threads_started":1}
I20260807 05:09:18.176898 9240 tablet_service.cc:1467] Processing CreateTablet for tablet 99ef1580135b4986b9e2c0e826b5c02d (DEFAULT_TABLE table=test-table [id=079da4714a3544d5ab59ceacd636d8d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:18.176994 8967 tablet_service.cc:1467] Processing CreateTablet for tablet 99ef1580135b4986b9e2c0e826b5c02d (DEFAULT_TABLE table=test-table [id=079da4714a3544d5ab59ceacd636d8d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:18.176994 9103 tablet_service.cc:1467] Processing CreateTablet for tablet 99ef1580135b4986b9e2c0e826b5c02d (DEFAULT_TABLE table=test-table [id=079da4714a3544d5ab59ceacd636d8d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:18.177906 9240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99ef1580135b4986b9e2c0e826b5c02d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:18.177983 8967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99ef1580135b4986b9e2c0e826b5c02d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:18.177983 9103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99ef1580135b4986b9e2c0e826b5c02d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:18.184298 9324 tablet_bootstrap.cc:492] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1: Bootstrap starting.
I20260807 05:09:18.185348 9326 tablet_bootstrap.cc:492] T 99ef1580135b4986b9e2c0e826b5c02d P dd163eb175a14f70919ceaff91fd797b: Bootstrap starting.
I20260807 05:09:18.185704 9325 tablet_bootstrap.cc:492] T 99ef1580135b4986b9e2c0e826b5c02d P 47d5246a1357428eb793c0b73dddedab: Bootstrap starting.
I20260807 05:09:18.186378 9324 tablet_bootstrap.cc:654] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:18.187348 9325 tablet_bootstrap.cc:654] T 99ef1580135b4986b9e2c0e826b5c02d P 47d5246a1357428eb793c0b73dddedab: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:18.187651 9324 log.cc:824] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:18.188455 9325 log.cc:824] T 99ef1580135b4986b9e2c0e826b5c02d P 47d5246a1357428eb793c0b73dddedab: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:18.189826 9324 tablet_bootstrap.cc:492] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1: No bootstrap required, opened a new log
I20260807 05:09:18.190014 9324 ts_tablet_manager.cc:1397] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 05:09:18.190201 9325 tablet_bootstrap.cc:492] T 99ef1580135b4986b9e2c0e826b5c02d P 47d5246a1357428eb793c0b73dddedab: No bootstrap required, opened a new log
I20260807 05:09:18.190367 9325 ts_tablet_manager.cc:1397] T 99ef1580135b4986b9e2c0e826b5c02d P 47d5246a1357428eb793c0b73dddedab: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:09:18.191115 9326 tablet_bootstrap.cc:654] T 99ef1580135b4986b9e2c0e826b5c02d P dd163eb175a14f70919ceaff91fd797b: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:18.191972 9326 log.cc:824] T 99ef1580135b4986b9e2c0e826b5c02d P dd163eb175a14f70919ceaff91fd797b: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:18.193542 9325 raft_consensus.cc:348] T 99ef1580135b4986b9e2c0e826b5c02d P 47d5246a1357428eb793c0b73dddedab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47d5246a1357428eb793c0b73dddedab" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 37625 } } peers { permanent_uuid: "dd163eb175a14f70919ceaff91fd797b" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37305 } } peers { permanent_uuid: "d7f7a43d89534957a1732dafed1e17e1" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 32845 } }
I20260807 05:09:18.193634 9324 raft_consensus.cc:348] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47d5246a1357428eb793c0b73dddedab" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 37625 } } peers { permanent_uuid: "dd163eb175a14f70919ceaff91fd797b" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37305 } } peers { permanent_uuid: "d7f7a43d89534957a1732dafed1e17e1" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 32845 } }
I20260807 05:09:18.193819 9325 raft_consensus.cc:374] T 99ef1580135b4986b9e2c0e826b5c02d P 47d5246a1357428eb793c0b73dddedab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:18.193837 9324 raft_consensus.cc:374] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:18.193882 9324 raft_consensus.cc:729] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f7a43d89534957a1732dafed1e17e1, State: Initialized, Role: FOLLOWER
I20260807 05:09:18.193886 9325 raft_consensus.cc:729] T 99ef1580135b4986b9e2c0e826b5c02d P 47d5246a1357428eb793c0b73dddedab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47d5246a1357428eb793c0b73dddedab, State: Initialized, Role: FOLLOWER
I20260807 05:09:18.194077 9326 tablet_bootstrap.cc:492] T 99ef1580135b4986b9e2c0e826b5c02d P dd163eb175a14f70919ceaff91fd797b: No bootstrap required, opened a new log
I20260807 05:09:18.194259 9326 ts_tablet_manager.cc:1397] T 99ef1580135b4986b9e2c0e826b5c02d P dd163eb175a14f70919ceaff91fd797b: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260807 05:09:18.194286 9324 consensus_queue.cc:260] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47d5246a1357428eb793c0b73dddedab" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 37625 } } peers { permanent_uuid: "dd163eb175a14f70919ceaff91fd797b" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37305 } } peers { permanent_uuid: "d7f7a43d89534957a1732dafed1e17e1" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 32845 } }
I20260807 05:09:18.194316 9325 consensus_queue.cc:260] T 99ef1580135b4986b9e2c0e826b5c02d P 47d5246a1357428eb793c0b73dddedab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47d5246a1357428eb793c0b73dddedab" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 37625 } } peers { permanent_uuid: "dd163eb175a14f70919ceaff91fd797b" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37305 } } peers { permanent_uuid: "d7f7a43d89534957a1732dafed1e17e1" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 32845 } }
I20260807 05:09:18.194895 9306 heartbeater.cc:503] Master 127.8.16.126:36453 was elected leader, sending a full tablet report...
I20260807 05:09:18.195221 9325 ts_tablet_manager.cc:1428] T 99ef1580135b4986b9e2c0e826b5c02d P 47d5246a1357428eb793c0b73dddedab: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 05:09:18.195257 9324 ts_tablet_manager.cc:1428] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 05:09:18.195515 9034 heartbeater.cc:503] Master 127.8.16.126:36453 was elected leader, sending a full tablet report...
I20260807 05:09:18.197827 9326 raft_consensus.cc:348] T 99ef1580135b4986b9e2c0e826b5c02d P dd163eb175a14f70919ceaff91fd797b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47d5246a1357428eb793c0b73dddedab" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 37625 } } peers { permanent_uuid: "dd163eb175a14f70919ceaff91fd797b" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37305 } } peers { permanent_uuid: "d7f7a43d89534957a1732dafed1e17e1" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 32845 } }
I20260807 05:09:18.198078 9326 raft_consensus.cc:374] T 99ef1580135b4986b9e2c0e826b5c02d P dd163eb175a14f70919ceaff91fd797b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:18.198136 9326 raft_consensus.cc:729] T 99ef1580135b4986b9e2c0e826b5c02d P dd163eb175a14f70919ceaff91fd797b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd163eb175a14f70919ceaff91fd797b, State: Initialized, Role: FOLLOWER
I20260807 05:09:18.198550 9326 consensus_queue.cc:260] T 99ef1580135b4986b9e2c0e826b5c02d P dd163eb175a14f70919ceaff91fd797b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47d5246a1357428eb793c0b73dddedab" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 37625 } } peers { permanent_uuid: "dd163eb175a14f70919ceaff91fd797b" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37305 } } peers { permanent_uuid: "d7f7a43d89534957a1732dafed1e17e1" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 32845 } }
I20260807 05:09:18.199138 9170 heartbeater.cc:503] Master 127.8.16.126:36453 was elected leader, sending a full tablet report...
I20260807 05:09:18.199486 9326 ts_tablet_manager.cc:1428] T 99ef1580135b4986b9e2c0e826b5c02d P dd163eb175a14f70919ceaff91fd797b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 05:09:18.213550 9330 raft_consensus.cc:482] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:09:18.213789 9330 raft_consensus.cc:504] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47d5246a1357428eb793c0b73dddedab" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 37625 } } peers { permanent_uuid: "dd163eb175a14f70919ceaff91fd797b" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37305 } } peers { permanent_uuid: "d7f7a43d89534957a1732dafed1e17e1" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 32845 } }
I20260807 05:09:18.214993 9330 leader_election.cc:288] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47d5246a1357428eb793c0b73dddedab (127.8.16.65:37625), dd163eb175a14f70919ceaff91fd797b (127.8.16.66:37305)
I20260807 05:09:18.219274 9319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:18.215288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:60049 (local address 127.8.16.65:37625)
0807 05:09:18.215407 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:18.215411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:18.215429 (+ 18us) server_negotiation.cc:407] Connection header received
0807 05:09:18.215513 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:18.215517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:18.215568 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:18.215659 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:18.215976 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.216637 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.218030 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.218657 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.218756 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:18.218822 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:18.219070 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 05:09:18.219149 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 05:09:18.219274 9308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:18.215203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:37625 (local address 127.8.16.67:60049)
0807 05:09:18.215317 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:18.215332 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:18.215421 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:18.215714 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:18.215718 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:18.215736 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:18.215841 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:18.215847 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.216746 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.216749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:18.217878 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:18.217891 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.218806 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.218810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:18.218934 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:18.218962 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 05:09:18.219039 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 05:09:18.219114 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 05:09:18.220053 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:18.215452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:37305 (local address 127.8.16.67:52225)
0807 05:09:18.215909 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:18.215922 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:18.216054 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:18.216291 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:18.216294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:18.216310 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:18.216420 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:18.216425 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.217272 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.217275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:18.218403 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:18.218415 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.219646 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.219651 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:18.219754 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:18.219777 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:09:18.219852 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 05:09:18.219917 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":151,"threads_started":1}
I20260807 05:09:18.220165 9323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:18.215600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:52225 (local address 127.8.16.66:37305)
0807 05:09:18.215744 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:18.215749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:18.215992 (+ 243us) server_negotiation.cc:407] Connection header received
0807 05:09:18.216160 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:18.216164 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:18.216211 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:18.216272 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:18.216512 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.217171 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.218543 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.219134 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.219230 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:18.219286 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:18.219899 (+ 613us) server_negotiation.cc:299] Negotiation successful
0807 05:09:18.220005 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 05:09:18.220116 8987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ef1580135b4986b9e2c0e826b5c02d" candidate_uuid: "d7f7a43d89534957a1732dafed1e17e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47d5246a1357428eb793c0b73dddedab" is_pre_election: true
I20260807 05:09:18.220605 8987 raft_consensus.cc:2454] T 99ef1580135b4986b9e2c0e826b5c02d P 47d5246a1357428eb793c0b73dddedab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7f7a43d89534957a1732dafed1e17e1 in term 0.
I20260807 05:09:18.220995 9123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ef1580135b4986b9e2c0e826b5c02d" candidate_uuid: "d7f7a43d89534957a1732dafed1e17e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd163eb175a14f70919ceaff91fd797b" is_pre_election: true
I20260807 05:09:18.221238 9123 raft_consensus.cc:2454] T 99ef1580135b4986b9e2c0e826b5c02d P dd163eb175a14f70919ceaff91fd797b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7f7a43d89534957a1732dafed1e17e1 in term 0.
I20260807 05:09:18.221256 9195 leader_election.cc:302] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47d5246a1357428eb793c0b73dddedab, d7f7a43d89534957a1732dafed1e17e1; no voters:
I20260807 05:09:18.221553 9330 raft_consensus.cc:2781] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:09:18.221627 9330 raft_consensus.cc:482] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:09:18.221665 9330 raft_consensus.cc:3037] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:18.222661 9330 raft_consensus.cc:504] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47d5246a1357428eb793c0b73dddedab" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 37625 } } peers { permanent_uuid: "dd163eb175a14f70919ceaff91fd797b" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37305 } } peers { permanent_uuid: "d7f7a43d89534957a1732dafed1e17e1" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 32845 } }
I20260807 05:09:18.222962 9330 leader_election.cc:288] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1 [CANDIDATE]: Term 1 election: Requested vote from peers 47d5246a1357428eb793c0b73dddedab (127.8.16.65:37625), dd163eb175a14f70919ceaff91fd797b (127.8.16.66:37305)
I20260807 05:09:18.223327 8987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ef1580135b4986b9e2c0e826b5c02d" candidate_uuid: "d7f7a43d89534957a1732dafed1e17e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47d5246a1357428eb793c0b73dddedab"
I20260807 05:09:18.223359 9123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ef1580135b4986b9e2c0e826b5c02d" candidate_uuid: "d7f7a43d89534957a1732dafed1e17e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd163eb175a14f70919ceaff91fd797b"
I20260807 05:09:18.223454 8987 raft_consensus.cc:3037] T 99ef1580135b4986b9e2c0e826b5c02d P 47d5246a1357428eb793c0b73dddedab [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:18.223454 9123 raft_consensus.cc:3037] T 99ef1580135b4986b9e2c0e826b5c02d P dd163eb175a14f70919ceaff91fd797b [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:18.224658 8987 raft_consensus.cc:2454] T 99ef1580135b4986b9e2c0e826b5c02d P 47d5246a1357428eb793c0b73dddedab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f7a43d89534957a1732dafed1e17e1 in term 1.
I20260807 05:09:18.224658 9123 raft_consensus.cc:2454] T 99ef1580135b4986b9e2c0e826b5c02d P dd163eb175a14f70919ceaff91fd797b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f7a43d89534957a1732dafed1e17e1 in term 1.
I20260807 05:09:18.224939 9195 leader_election.cc:302] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7f7a43d89534957a1732dafed1e17e1, dd163eb175a14f70919ceaff91fd797b; no voters:
I20260807 05:09:18.225107 9330 raft_consensus.cc:2781] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:18.225334 9330 raft_consensus.cc:686] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1 [term 1 LEADER]: Becoming Leader. State: Replica: d7f7a43d89534957a1732dafed1e17e1, State: Running, Role: LEADER
I20260807 05:09:18.225692 9330 consensus_queue.cc:237] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47d5246a1357428eb793c0b73dddedab" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 37625 } } peers { permanent_uuid: "dd163eb175a14f70919ceaff91fd797b" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37305 } } peers { permanent_uuid: "d7f7a43d89534957a1732dafed1e17e1" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 32845 } }
I20260807 05:09:18.228873 8839 catalog_manager.cc:5495] T 99ef1580135b4986b9e2c0e826b5c02d P d7f7a43d89534957a1732dafed1e17e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7f7a43d89534957a1732dafed1e17e1 (127.8.16.67). New cstate: current_term: 1 leader_uuid: "d7f7a43d89534957a1732dafed1e17e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47d5246a1357428eb793c0b73dddedab" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 37625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd163eb175a14f70919ceaff91fd797b" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7f7a43d89534957a1732dafed1e17e1" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 32845 } health_report { overall_health: HEALTHY } } }
Aug 07 05:09:18 dist-test-slave-49h3 krb5kdc[8794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.65@KRBTEST.COM
I20260807 05:09:18.253897 9319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:18.245545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36478 (local address 127.8.16.65:37625)
0807 05:09:18.245673 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:18.245678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:18.245843 (+ 165us) server_negotiation.cc:407] Connection header received
0807 05:09:18.245981 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:18.245983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:18.246027 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:18.246102 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:18.246784 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.247285 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.248121 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.248337 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.248425 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:18.250747 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:18.250760 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:18.250775 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:18.250800 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:18.252202 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:18.252597 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:18.252602 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:18.252604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:18.252703 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:18.252948 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:18.252951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:18.252953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:18.253201 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:18.253428 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:18.253626 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 05:09:18.253747 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
Aug 07 05:09:18 dist-test-slave-49h3 krb5kdc[8794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079358, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 05:09:18 dist-test-slave-49h3 krb5kdc[8794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079358, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.16.65@KRBTEST.COM
I20260807 05:09:18.279888 9319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:18.272211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36484 (local address 127.8.16.65:37625)
0807 05:09:18.272329 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:18.272332 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:18.272509 (+ 177us) server_negotiation.cc:407] Connection header received
0807 05:09:18.272619 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:18.272621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:18.272659 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:18.272729 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:18.273312 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.273862 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.274630 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.274846 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.274922 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:18.277121 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:18.277124 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:18.277126 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:18.277149 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:18.278356 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:18.278743 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:18.278747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:18.278749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:18.278826 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:18.279062 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:18.279065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:18.279067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:18.279222 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:18.279350 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:18.279628 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 05:09:18.279743 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Aug 07 05:09:18 dist-test-slave-49h3 krb5kdc[8794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 05:09:18 dist-test-slave-49h3 krb5kdc[8794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.65@KRBTEST.COM
I20260807 05:09:18.304329 9319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:18.296611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36486 (local address 127.8.16.65:37625)
0807 05:09:18.296760 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:18.296764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:18.296878 (+ 114us) server_negotiation.cc:407] Connection header received
0807 05:09:18.297007 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:18.297010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:18.297057 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:18.297128 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:18.297841 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.298338 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.299127 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:18.299346 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:18.299431 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:18.301634 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:18.301637 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:18.301638 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:18.301669 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:18.302949 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:18.303357 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:18.303360 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:18.303361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:18.303435 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:18.303697 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:18.303699 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:18.303700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:18.303851 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:18.303943 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:18.304111 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 05:09:18.304206 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 05:09:18.305691 8947 tablet_service.cc:2908] Rejecting scan request for tablet 99ef1580135b4986b9e2c0e826b5c02d: Uninitialized: clean time has not yet been initialized
I20260807 05:09:18.307807 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 8903
I20260807 05:09:18.314059 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 9039
I20260807 05:09:18.320087 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 9175
I20260807 05:09:18.325253 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 8807
2026-08-07T05:09:18Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2153 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:09:18 dist-test-slave-49h3 krb5kdc[9348](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:09:18 dist-test-slave-49h3 krb5kdc[9348](info): set up 2 sockets
Aug 07 05:09:18 dist-test-slave-49h3 krb5kdc[9348](info): commencing operation
krb5kdc: starting...
W20260807 05:09:19.317260 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.971s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:09:19 dist-test-slave-49h3 krb5kdc[9348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:09:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:09:19.434288 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:35739
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:35393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:35739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:19.541628 9361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:19.541896 9361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:19.541940 9361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:19.545998 9361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:19.546088 9361 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 05:09:19.546104 9361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:19.546126 9361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:19.546156 9361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:19.546181 9361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:19.551671 9361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:35393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:35739
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:35739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:19.552778 9361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:19.553792 9361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:19.561877 9366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:19.561877 9369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:19.561919 9367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:19.562194 9361 server_base.cc:1034] running on GCE node
I20260807 05:09:19.562812 9361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:19.563936 9361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:19.565093 9361 hybrid_clock.cc:648] HybridClock initialized: now 1786079359565068 us; error 42 us; skew 500 ppm
Aug 07 05:09:19 dist-test-slave-49h3 krb5kdc[9348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079359, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:19.567842 9361 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:09:19.568763 9361 webserver.cc:460] Webserver started at http://127.8.16.126:35685/ using document root <none> and password file <none>
I20260807 05:09:19.569296 9361 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:19.569358 9361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:19.569577 9361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:19.571686 9361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "13fd3c6b893b4878ac63c3f251b309fc"
format_stamp: "Formatted at 2026-08-07 05:09:19 on dist-test-slave-49h3"
server_key: "e202b1633a94ed7778287538b2b7819b"
server_key_iv: "0e124c2efb828d76a83cbfe7cfd9bbf5"
server_key_version: "encryptionkey@0"
I20260807 05:09:19.572260 9361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "13fd3c6b893b4878ac63c3f251b309fc"
format_stamp: "Formatted at 2026-08-07 05:09:19 on dist-test-slave-49h3"
server_key: "e202b1633a94ed7778287538b2b7819b"
server_key_iv: "0e124c2efb828d76a83cbfe7cfd9bbf5"
server_key_version: "encryptionkey@0"
I20260807 05:09:19.576005 9361 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:09:19.578344 9376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:19.579532 9361 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:19.579636 9361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "13fd3c6b893b4878ac63c3f251b309fc"
format_stamp: "Formatted at 2026-08-07 05:09:19 on dist-test-slave-49h3"
server_key: "e202b1633a94ed7778287538b2b7819b"
server_key_iv: "0e124c2efb828d76a83cbfe7cfd9bbf5"
server_key_version: "encryptionkey@0"
I20260807 05:09:19.579751 9361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:19.598918 9361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:19.602011 9361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:19.602192 9361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:19.609551 9361 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:35739
I20260807 05:09:19.609581 9439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:35739 every 8 connection(s)
I20260807 05:09:19.610513 9361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:19.613164 9440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:19.618682 9440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc: Bootstrap starting.
I20260807 05:09:19.620723 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 9361
I20260807 05:09:19.620818 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:09:19.621055 8257 external_mini_cluster.cc:1428] Setting key c8289b4910bec75d52025f12989dabb1
I20260807 05:09:19.621091 9440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:19.621984 9440 log.cc:824] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:19.624009 9440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc: No bootstrap required, opened a new log
I20260807 05:09:19.628161 9440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fd3c6b893b4878ac63c3f251b309fc" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35739 } }
I20260807 05:09:19.628459 9440 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:19.628521 9440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13fd3c6b893b4878ac63c3f251b309fc, State: Initialized, Role: FOLLOWER
I20260807 05:09:19.629045 9440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fd3c6b893b4878ac63c3f251b309fc" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35739 } }
I20260807 05:09:19.629204 9440 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:09:19.629277 9440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:09:19.629371 9440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc [term 0 FOLLOWER]: Advancing to term 1
Aug 07 05:09:19 dist-test-slave-49h3 krb5kdc[9348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:19.630453 9440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fd3c6b893b4878ac63c3f251b309fc" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35739 } }
I20260807 05:09:19.630782 9440 leader_election.cc:302] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13fd3c6b893b4878ac63c3f251b309fc; no voters:
I20260807 05:09:19.631131 9440 leader_election.cc:288] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:09:19.631218 9445 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:19.631670 9445 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc [term 1 LEADER]: Becoming Leader. State: Replica: 13fd3c6b893b4878ac63c3f251b309fc, State: Running, Role: LEADER
I20260807 05:09:19.631963 9445 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fd3c6b893b4878ac63c3f251b309fc" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35739 } }
I20260807 05:09:19.632541 9440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:19.633047 9445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13fd3c6b893b4878ac63c3f251b309fc. Latest consensus state: current_term: 1 leader_uuid: "13fd3c6b893b4878ac63c3f251b309fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fd3c6b893b4878ac63c3f251b309fc" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35739 } } }
I20260807 05:09:19.633158 9445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:19.633635 9446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13fd3c6b893b4878ac63c3f251b309fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fd3c6b893b4878ac63c3f251b309fc" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35739 } } }
I20260807 05:09:19.633737 9446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:19.634363 9450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:19.635730 9444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:19.622171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60488 (local address 127.8.16.126:35739)
0807 05:09:19.622590 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:19.622610 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:19.622653 (+ 43us) server_negotiation.cc:407] Connection header received
0807 05:09:19.623337 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:19.623353 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:19.623668 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:19.623944 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:19.624826 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:19.625904 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:19.627071 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:19.627551 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:19.627665 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:19.630441 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:19.630447 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:19.630461 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:19.630487 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:19.632443 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:19.632965 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:19.632971 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:19.632977 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:19.633133 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:19.633465 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:19.633469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:19.633471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:19.633937 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:19.634137 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:19.634476 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 05:09:19.634789 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":143,"threads_started":1}
I20260807 05:09:19.637742 9450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:19.643162 9450 catalog_manager.cc:1347] Generated new cluster ID: 84cee33f487247ef928de1ac5dd6481b
I20260807 05:09:19.643213 9450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:19.664727 9450 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:09:19.666114 9450 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:19.673420 9450 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc: Generated new TSK 0
I20260807 05:09:19.674145 9450 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
I20260807 05:09:19.704679 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:35739
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:35393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:19.815824 9466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:19.816093 9466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:19.816146 9466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:19.820153 9466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:19.820230 9466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:19.820292 9466 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 05:09:19.820348 9466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:09:19.825598 9466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:35393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.16.126:35739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:19.826673 9466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:19.827680 9466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:19.837182 9474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:19.837260 9466 server_base.cc:1034] running on GCE node
W20260807 05:09:19.837301 9471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:19.837182 9472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:19.837827 9466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:19.838451 9466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:19.839622 9466 hybrid_clock.cc:648] HybridClock initialized: now 1786079359839598 us; error 32 us; skew 500 ppm
Aug 07 05:09:19 dist-test-slave-49h3 krb5kdc[9348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079359, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:19.842320 9466 init.cc:377] Logged in from keytab as kudu/127.8.16.65@KRBTEST.COM (short username kudu)
I20260807 05:09:19.843174 9466 webserver.cc:460] Webserver started at http://127.8.16.65:45511/ using document root <none> and password file <none>
I20260807 05:09:19.843675 9466 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:19.843734 9466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:19.843901 9466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:19.845952 9466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "3086396ef63f464db53ccef657279217"
format_stamp: "Formatted at 2026-08-07 05:09:19 on dist-test-slave-49h3"
server_key: "6113c94d8fa9fbc3c3f88a9e70ecad99"
server_key_iv: "82929e185b0b7ab7f94ccaf231de144c"
server_key_version: "encryptionkey@0"
I20260807 05:09:19.846508 9466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "3086396ef63f464db53ccef657279217"
format_stamp: "Formatted at 2026-08-07 05:09:19 on dist-test-slave-49h3"
server_key: "6113c94d8fa9fbc3c3f88a9e70ecad99"
server_key_iv: "82929e185b0b7ab7f94ccaf231de144c"
server_key_version: "encryptionkey@0"
I20260807 05:09:19.850028 9466 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:09:19.852401 9481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:19.853670 9466 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:19.853787 9466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "3086396ef63f464db53ccef657279217"
format_stamp: "Formatted at 2026-08-07 05:09:19 on dist-test-slave-49h3"
server_key: "6113c94d8fa9fbc3c3f88a9e70ecad99"
server_key_iv: "82929e185b0b7ab7f94ccaf231de144c"
server_key_version: "encryptionkey@0"
I20260807 05:09:19.853899 9466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:19.875665 9466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:19.878474 9466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:19.878631 9466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:19.880038 9466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:19.880105 9466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:19.880151 9466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:19.880180 9466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 05:09:19 dist-test-slave-49h3 krb5kdc[9348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079359, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:19.890070 9466 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:34607
I20260807 05:09:19.890934 9466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:09:19.891433 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 9466
I20260807 05:09:19.891554 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:09:19.891851 8257 external_mini_cluster.cc:1428] Setting key 4b39e367a583d1e9e9d2a0b45ac687b3
I20260807 05:09:19.893024 9444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:19.880746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:60313 (local address 127.8.16.126:35739)
0807 05:09:19.880889 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:19.880894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:19.881776 (+ 882us) server_negotiation.cc:407] Connection header received
0807 05:09:19.882747 (+ 971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:19.882752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:19.882811 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:19.882932 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:19.884205 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:19.884845 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:19.886214 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:19.886499 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:19.886593 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:19.889310 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:19.889313 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:19.889315 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:19.889340 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:19.890786 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:19.891388 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:19.891392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:19.891395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:19.891481 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:19.891874 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:19.891878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:19.891880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:19.892113 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:19.892205 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:19.892755 (+ 550us) server_negotiation.cc:299] Negotiation successful
0807 05:09:19.892876 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 05:09:19.893774 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:19.881143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:35739 (local address 127.8.16.65:60313)
0807 05:09:19.881616 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:19.881667 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:19.882498 (+ 831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:19.883095 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:19.883102 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:19.883530 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:19.884063 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:19.884084 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:19.884966 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:19.884969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:19.886098 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:19.886109 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:19.886654 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:19.886658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:19.886815 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:19.887396 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:19.887412 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:19.889146 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:19.890893 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:19.890899 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:19.890912 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:19.891261 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:19.891581 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:19.891584 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:19.891586 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:19.891763 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:19.892252 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:19.892257 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:19.892511 (+ 254us) client_negotiation.cc:769] Sending connection context
0807 05:09:19.892689 (+ 178us) client_negotiation.cc:240] Negotiation successful
0807 05:09:19.892929 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":113,"threads_started":1}
I20260807 05:09:19.891676 9599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:34607 every 8 connection(s)
I20260807 05:09:19.895038 9600 heartbeater.cc:347] Connected to a master server at 127.8.16.126:35739
I20260807 05:09:19.895193 9600 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:19.895682 9600 heartbeater.cc:511] Master 127.8.16.126:35739 requested a full tablet report, sending...
I20260807 05:09:19.897198 9392 ts_manager.cc:194] Registered new tserver with Master: 3086396ef63f464db53ccef657279217 (127.8.16.65:34607)
I20260807 05:09:19.898659 9392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:60313
WARNING: no policy specified for kudu/127.8.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.66@KRBTEST.COM" created.
I20260807 05:09:19.905074 9444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:19.896829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:52509 (local address 127.8.16.126:35739)
0807 05:09:19.897557 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:19.897561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:19.897798 (+ 237us) server_negotiation.cc:407] Connection header received
0807 05:09:19.897908 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:19.897911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:19.897953 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:19.898020 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:19.898474 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:19.899221 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:19.901142 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:19.901493 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:19.901608 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:19.902593 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:19.902606 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:19.902609 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:19.902634 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:19.903783 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:19.904129 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:19.904131 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:19.904132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:19.904195 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:19.904403 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:19.904405 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:19.904406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:19.904550 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:19.904627 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:19.904818 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 05:09:19.904926 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655}
I20260807 05:09:19.905082 9607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:19.896762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:35739 (local address 127.8.16.65:52509)
0807 05:09:19.897712 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:19.897726 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:19.897818 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:19.898048 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:19.898051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:19.898255 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:19.898364 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:19.898370 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:19.899343 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:19.899346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:19.901032 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:19.901041 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:19.901592 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:19.901595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:19.901669 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:19.902150 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:19.902163 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:19.902464 (+ 301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:19.903874 (+ 1410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:19.903876 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:19.903878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:19.904058 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:19.904252 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:19.904254 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:19.904255 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:19.904334 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:19.904613 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:19.904615 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:19.904698 (+ 83us) client_negotiation.cc:769] Sending connection context
0807 05:09:19.904779 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 05:09:19.904908 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"thread_start_us":91,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
I20260807 05:09:19.919710 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:35739
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:35393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:20.027441 9609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:20.027715 9609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:20.027760 9609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:20.031770 9609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:20.031839 9609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:20.031880 9609 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 05:09:20.031932 9609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:09:20.037407 9609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:35393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.16.126:35739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:20.038512 9609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:20.039526 9609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:20.048820 9617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:20.048818 9614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:20.048884 9609 server_base.cc:1034] running on GCE node
W20260807 05:09:20.048815 9615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:20.049508 9609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:20.050132 9609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:20.051316 9609 hybrid_clock.cc:648] HybridClock initialized: now 1786079360051298 us; error 27 us; skew 500 ppm
Aug 07 05:09:20 dist-test-slave-49h3 krb5kdc[9348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079360, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:20.054055 9609 init.cc:377] Logged in from keytab as kudu/127.8.16.66@KRBTEST.COM (short username kudu)
I20260807 05:09:20.054929 9609 webserver.cc:460] Webserver started at http://127.8.16.66:38631/ using document root <none> and password file <none>
I20260807 05:09:20.055444 9609 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:20.055500 9609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:20.055670 9609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:20.057775 9609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "b6ac299e54b8400fa308a5a99c053333"
format_stamp: "Formatted at 2026-08-07 05:09:20 on dist-test-slave-49h3"
server_key: "c3ccd97a3e652e90b7ed1509613071c5"
server_key_iv: "bce8256d0eef57f12fe148388106508d"
server_key_version: "encryptionkey@0"
I20260807 05:09:20.058293 9609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "b6ac299e54b8400fa308a5a99c053333"
format_stamp: "Formatted at 2026-08-07 05:09:20 on dist-test-slave-49h3"
server_key: "c3ccd97a3e652e90b7ed1509613071c5"
server_key_iv: "bce8256d0eef57f12fe148388106508d"
server_key_version: "encryptionkey@0"
I20260807 05:09:20.061848 9609 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:09:20.064143 9624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:20.065260 9609 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:20.065375 9609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "b6ac299e54b8400fa308a5a99c053333"
format_stamp: "Formatted at 2026-08-07 05:09:20 on dist-test-slave-49h3"
server_key: "c3ccd97a3e652e90b7ed1509613071c5"
server_key_iv: "bce8256d0eef57f12fe148388106508d"
server_key_version: "encryptionkey@0"
I20260807 05:09:20.065500 9609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:20.086273 9609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:20.088967 9609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:20.089141 9609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:20.090571 9609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:20.090621 9609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:20.090673 9609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:20.090698 9609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 05:09:20 dist-test-slave-49h3 krb5kdc[9348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079360, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:20.100653 9609 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:32779
I20260807 05:09:20.100925 9740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:32779 every 8 connection(s)
I20260807 05:09:20.101629 9609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:09:20.103665 9444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.091239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:59853 (local address 127.8.16.126:35739)
0807 05:09:20.091358 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:20.091363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:20.092361 (+ 998us) server_negotiation.cc:407] Connection header received
0807 05:09:20.093381 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:20.093385 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:20.093470 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:20.093528 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:20.094914 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.095585 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.096861 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.097132 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.097227 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.099958 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:20.099962 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:20.099964 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:20.099988 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:20.101633 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:20.102237 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:20.102241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:20.102243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:20.102337 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:20.102638 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:20.102641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:20.102643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:20.102826 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:20.102910 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:20.103373 (+ 463us) server_negotiation.cc:299] Negotiation successful
0807 05:09:20.103494 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 05:09:20.104380 9633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.091749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:35739 (local address 127.8.16.66:59853)
0807 05:09:20.092212 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:20.092252 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:20.093133 (+ 881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:20.093896 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:20.093903 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:20.094216 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:20.094764 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.094785 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.095703 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.095707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.096745 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.096754 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.097288 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.097292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.097501 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.098054 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:20.098071 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:20.099779 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:20.101762 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:20.101768 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:20.101781 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:20.102116 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:20.102411 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:20.102414 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:20.102417 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:20.102548 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:20.102927 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:20.102933 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:20.103161 (+ 228us) client_negotiation.cc:769] Sending connection context
0807 05:09:20.103316 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 05:09:20.103609 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":137,"threads_started":1}
I20260807 05:09:20.105643 9741 heartbeater.cc:347] Connected to a master server at 127.8.16.126:35739
I20260807 05:09:20.105769 9741 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:20.106267 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 9609
I20260807 05:09:20.106329 9741 heartbeater.cc:511] Master 127.8.16.126:35739 requested a full tablet report, sending...
I20260807 05:09:20.106380 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:09:20.106673 8257 external_mini_cluster.cc:1428] Setting key e9e6f350144f04ba9dc73f234b1a5bef
I20260807 05:09:20.107368 9392 ts_manager.cc:194] Registered new tserver with Master: b6ac299e54b8400fa308a5a99c053333 (127.8.16.66:32779)
I20260807 05:09:20.108026 9392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:59853
I20260807 05:09:20.115443 9747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.107880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:35739 (local address 127.8.16.66:60395)
0807 05:09:20.108149 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:20.108164 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:20.108240 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:20.109756 (+ 1516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:20.109759 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:20.109939 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:20.110035 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.110040 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.110776 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.110779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.111330 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.111337 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.111739 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.111743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.111826 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.112260 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:20.112271 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:20.112577 (+ 306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:20.114080 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:20.114084 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:20.114086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:20.114274 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:20.114502 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:20.114505 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:20.114507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:20.114583 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:20.114948 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:20.114952 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:20.115038 (+ 86us) client_negotiation.cc:769] Sending connection context
0807 05:09:20.115129 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 05:09:20.115263 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":86,"threads_started":1}
I20260807 05:09:20.115439 9444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.107906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:60395 (local address 127.8.16.126:35739)
0807 05:09:20.109576 (+ 1670us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:20.109579 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:20.109597 (+ 18us) server_negotiation.cc:407] Connection header received
0807 05:09:20.109625 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:20.109626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:20.109658 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:20.109722 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:20.110155 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.110661 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.111429 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.111644 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.111726 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.112697 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:20.112701 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:20.112703 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:20.112727 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:20.113983 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:20.114355 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:20.114358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:20.114360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:20.114426 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:20.114657 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:20.114661 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:20.114662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:20.114858 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:20.114944 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:20.115191 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 05:09:20.115289 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1601}
WARNING: no policy specified for kudu/127.8.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
I20260807 05:09:20.132166 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:35739
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:35393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:20.241964 9750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:20.242220 9750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:20.242264 9750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:20.246291 9750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:20.246349 9750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:20.246388 9750 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 05:09:20.246439 9750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:09:20.251808 9750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:35393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.16.126:35739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:20.252839 9750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:20.253866 9750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 05:09:20.263235 9750 server_base.cc:1034] running on GCE node
W20260807 05:09:20.263195 9755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:20.263199 9756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:20.263201 9758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:20.263805 9750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:20.264427 9750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:20.265597 9750 hybrid_clock.cc:648] HybridClock initialized: now 1786079360265578 us; error 28 us; skew 500 ppm
Aug 07 05:09:20 dist-test-slave-49h3 krb5kdc[9348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079360, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:20.268333 9750 init.cc:377] Logged in from keytab as kudu/127.8.16.67@KRBTEST.COM (short username kudu)
I20260807 05:09:20.269178 9750 webserver.cc:460] Webserver started at http://127.8.16.67:34031/ using document root <none> and password file <none>
I20260807 05:09:20.269750 9750 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:20.269812 9750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:20.270017 9750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:20.272006 9750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "12831c9e6cc0414fafe6350b67a3aba6"
format_stamp: "Formatted at 2026-08-07 05:09:20 on dist-test-slave-49h3"
server_key: "d80001ddf3b49b98e909853496780f76"
server_key_iv: "4dc8cbe1c1d121464584747351c6d2bc"
server_key_version: "encryptionkey@0"
I20260807 05:09:20.272524 9750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "12831c9e6cc0414fafe6350b67a3aba6"
format_stamp: "Formatted at 2026-08-07 05:09:20 on dist-test-slave-49h3"
server_key: "d80001ddf3b49b98e909853496780f76"
server_key_iv: "4dc8cbe1c1d121464584747351c6d2bc"
server_key_version: "encryptionkey@0"
I20260807 05:09:20.276110 9750 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:09:20.278374 9765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:20.279498 9750 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:20.279594 9750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "12831c9e6cc0414fafe6350b67a3aba6"
format_stamp: "Formatted at 2026-08-07 05:09:20 on dist-test-slave-49h3"
server_key: "d80001ddf3b49b98e909853496780f76"
server_key_iv: "4dc8cbe1c1d121464584747351c6d2bc"
server_key_version: "encryptionkey@0"
I20260807 05:09:20.279704 9750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:20.303313 9750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:20.306346 9750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:20.306528 9750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:20.307976 9750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:20.308028 9750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:20.308068 9750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:20.308096 9750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 05:09:20 dist-test-slave-49h3 krb5kdc[9348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079360, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:20.318065 9750 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:38771
I20260807 05:09:20.318149 9881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:38771 every 8 connection(s)
I20260807 05:09:20.318920 9750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:09:20.321007 9444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.308805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:44479 (local address 127.8.16.126:35739)
0807 05:09:20.308916 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:20.308920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:20.310024 (+ 1104us) server_negotiation.cc:407] Connection header received
0807 05:09:20.311057 (+ 1033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:20.311062 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:20.311125 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:20.311235 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:20.312694 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.313538 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.314794 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.315022 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.315112 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.317552 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:20.317555 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:20.317557 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:20.317580 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:20.319049 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:20.319618 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:20.319622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:20.319624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:20.319712 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:20.320015 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:20.320018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:20.320020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:20.320198 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:20.320281 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:20.320730 (+ 449us) server_negotiation.cc:299] Negotiation successful
0807 05:09:20.320846 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 05:09:20.321820 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.309361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:35739 (local address 127.8.16.67:44479)
0807 05:09:20.309869 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:20.309913 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:20.310825 (+ 912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:20.311446 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:20.311455 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:20.311872 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:20.312516 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.312534 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.313670 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.313673 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.314676 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.314686 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.315168 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.315171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.315348 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.315894 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:20.315910 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:20.317373 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:20.319156 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:20.319162 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:20.319175 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:20.319494 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:20.319795 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:20.319798 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:20.319801 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:20.319929 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:20.320290 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:20.320297 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:20.320515 (+ 218us) client_negotiation.cc:769] Sending connection context
0807 05:09:20.320672 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 05:09:20.320972 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":139,"threads_started":1}
I20260807 05:09:20.322928 9882 heartbeater.cc:347] Connected to a master server at 127.8.16.126:35739
I20260807 05:09:20.323064 9882 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:20.323551 9882 heartbeater.cc:511] Master 127.8.16.126:35739 requested a full tablet report, sending...
I20260807 05:09:20.324522 9393 ts_manager.cc:194] Registered new tserver with Master: 12831c9e6cc0414fafe6350b67a3aba6 (127.8.16.67:38771)
I20260807 05:09:20.325145 9393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:44479
I20260807 05:09:20.329023 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 9750
I20260807 05:09:20.329126 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:09:20.329361 8257 external_mini_cluster.cc:1428] Setting key f22a2bf7d99eb1b2c323af1ebc52255c
I20260807 05:09:20.331246 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:09:20.331432 9444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.324860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:33037 (local address 127.8.16.126:35739)
0807 05:09:20.324977 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:20.324981 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:20.325174 (+ 193us) server_negotiation.cc:407] Connection header received
0807 05:09:20.325270 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:20.325273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:20.325314 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:20.325376 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:20.325803 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.326325 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.327102 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.327314 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.327395 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.328362 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:20.328366 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:20.328369 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:20.328392 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:20.329883 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:20.330361 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:20.330364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:20.330366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:20.330445 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:20.330693 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:20.330696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:20.330697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:20.330863 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:20.330946 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:20.331196 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 05:09:20.331296 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 05:09:20.331450 9888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.324776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:35739 (local address 127.8.16.67:33037)
0807 05:09:20.325098 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:20.325109 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:20.325196 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:20.325401 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:20.325404 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:20.325610 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:20.325705 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.325710 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.326437 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.326440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.326986 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.326994 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.327408 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.327412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.327497 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.327939 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:20.327950 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:20.328240 (+ 290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:20.330079 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:20.330081 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:20.330083 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:20.330271 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:20.330524 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:20.330527 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:20.330529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:20.330606 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:20.330950 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:20.330953 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:20.331040 (+ 87us) client_negotiation.cc:769] Sending connection context
0807 05:09:20.331123 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 05:09:20.331264 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":92,"threads_started":1}
Aug 07 05:09:20 dist-test-slave-49h3 krb5kdc[9348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079360, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 05:09:20 dist-test-slave-49h3 krb5kdc[9348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079360, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:20.356559 9444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.348997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60502 (local address 127.8.16.126:35739)
0807 05:09:20.349139 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:20.349142 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:20.349155 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:09:20.349197 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:20.349199 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:20.349238 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:20.349302 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:20.350032 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.350589 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.351377 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.351595 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.351676 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.353998 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:20.354001 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:20.354003 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:20.354029 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:20.355320 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:20.355673 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:20.355675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:20.355677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:20.355743 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:20.355917 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:20.355919 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:20.355921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:20.356073 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:20.356180 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:20.356324 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 05:09:20.356409 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 05:09:20.359272 9393 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:09:20.362200 9393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:09:20.375527 9905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.370769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33624 (local address 127.8.16.65:34607)
0807 05:09:20.371279 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:20.371295 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:20.371315 (+ 20us) server_negotiation.cc:407] Connection header received
0807 05:09:20.371385 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:20.371390 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:20.371565 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:20.371706 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:20.371932 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.372734 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.374261 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.374892 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.374988 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.375154 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:20.375290 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 05:09:20.375376 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":97,"threads_started":1}
I20260807 05:09:20.375511 9901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.370320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:34607 (local address 127.0.0.1:33624)
0807 05:09:20.371161 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:20.371173 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:20.371240 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:20.371703 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:20.371706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:20.371721 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:20.371813 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.371819 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.372842 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.372845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.374115 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.374125 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.375001 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.375005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.375108 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.375146 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 05:09:20.375250 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 05:09:20.375369 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"thread_start_us":75,"threads_started":1}
I20260807 05:09:20.375756 9903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.370088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:38771 (local address 127.0.0.1:42916)
0807 05:09:20.370822 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:20.370836 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:20.370959 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:20.371873 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:20.371876 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:20.371892 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:20.371958 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.371963 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.373230 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.373233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.374402 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.374411 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.375382 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.375386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.375484 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.375505 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 05:09:20.375552 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 05:09:20.375615 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":2,"thread_start_us":110,"threads_started":1}
I20260807 05:09:20.376051 9902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.370106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39338 (local address 127.8.16.66:32779)
0807 05:09:20.370508 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:20.370519 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:20.370886 (+ 367us) server_negotiation.cc:407] Connection header received
0807 05:09:20.371129 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:20.371134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:20.371345 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:20.371486 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:20.371925 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.373081 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.374394 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.375198 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.375304 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.375456 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:20.375824 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 05:09:20.375917 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":109,"threads_started":1}
I20260807 05:09:20.376300 9900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.369974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:32779 (local address 127.0.0.1:39338)
0807 05:09:20.370745 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:20.370794 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:20.370962 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:20.371648 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:20.371653 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:20.371674 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:20.371800 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.371809 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.373221 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.373224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.374286 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.374294 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.375507 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.375510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.375596 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.375626 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 05:09:20.375685 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 05:09:20.375736 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":40,"thread_start_us":67,"threads_started":1}
I20260807 05:09:20.376444 9904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.370741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42916 (local address 127.8.16.67:38771)
0807 05:09:20.371476 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:20.371488 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:20.371507 (+ 19us) server_negotiation.cc:407] Connection header received
0807 05:09:20.371573 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:20.371577 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:20.371757 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:20.371901 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:20.372041 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.373128 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.374519 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.375260 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.376018 (+ 758us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.376188 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:20.376236 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 05:09:20.376310 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":97,"threads_started":1}
I20260807 05:09:20.378729 9675 tablet_service.cc:1467] Processing CreateTablet for tablet 8b6a0f58c52f461bbddafa53ef95a98b (DEFAULT_TABLE table=test-table [id=312f6893df1e4f35a7c56d5be70ea7e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:20.378844 9816 tablet_service.cc:1467] Processing CreateTablet for tablet 8b6a0f58c52f461bbddafa53ef95a98b (DEFAULT_TABLE table=test-table [id=312f6893df1e4f35a7c56d5be70ea7e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:20.378844 9531 tablet_service.cc:1467] Processing CreateTablet for tablet 8b6a0f58c52f461bbddafa53ef95a98b (DEFAULT_TABLE table=test-table [id=312f6893df1e4f35a7c56d5be70ea7e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:20.379669 9675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b6a0f58c52f461bbddafa53ef95a98b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:20.379812 9816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b6a0f58c52f461bbddafa53ef95a98b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:20.379805 9531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b6a0f58c52f461bbddafa53ef95a98b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:20.386543 9906 tablet_bootstrap.cc:492] T 8b6a0f58c52f461bbddafa53ef95a98b P b6ac299e54b8400fa308a5a99c053333: Bootstrap starting.
I20260807 05:09:20.387851 9907 tablet_bootstrap.cc:492] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217: Bootstrap starting.
I20260807 05:09:20.387907 9908 tablet_bootstrap.cc:492] T 8b6a0f58c52f461bbddafa53ef95a98b P 12831c9e6cc0414fafe6350b67a3aba6: Bootstrap starting.
I20260807 05:09:20.388861 9906 tablet_bootstrap.cc:654] T 8b6a0f58c52f461bbddafa53ef95a98b P b6ac299e54b8400fa308a5a99c053333: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:20.389477 9908 tablet_bootstrap.cc:654] T 8b6a0f58c52f461bbddafa53ef95a98b P 12831c9e6cc0414fafe6350b67a3aba6: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:20.389773 9907 tablet_bootstrap.cc:654] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:20.389761 9906 log.cc:824] T 8b6a0f58c52f461bbddafa53ef95a98b P b6ac299e54b8400fa308a5a99c053333: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:20.390161 9908 log.cc:824] T 8b6a0f58c52f461bbddafa53ef95a98b P 12831c9e6cc0414fafe6350b67a3aba6: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:20.390486 9907 log.cc:824] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:20.391808 9906 tablet_bootstrap.cc:492] T 8b6a0f58c52f461bbddafa53ef95a98b P b6ac299e54b8400fa308a5a99c053333: No bootstrap required, opened a new log
I20260807 05:09:20.392000 9906 ts_tablet_manager.cc:1397] T 8b6a0f58c52f461bbddafa53ef95a98b P b6ac299e54b8400fa308a5a99c053333: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 05:09:20.392390 9907 tablet_bootstrap.cc:492] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217: No bootstrap required, opened a new log
I20260807 05:09:20.392460 9908 tablet_bootstrap.cc:492] T 8b6a0f58c52f461bbddafa53ef95a98b P 12831c9e6cc0414fafe6350b67a3aba6: No bootstrap required, opened a new log
I20260807 05:09:20.392635 9908 ts_tablet_manager.cc:1397] T 8b6a0f58c52f461bbddafa53ef95a98b P 12831c9e6cc0414fafe6350b67a3aba6: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 05:09:20.392665 9907 ts_tablet_manager.cc:1397] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:09:20.395529 9908 raft_consensus.cc:348] T 8b6a0f58c52f461bbddafa53ef95a98b P 12831c9e6cc0414fafe6350b67a3aba6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } } peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } }
I20260807 05:09:20.395750 9908 raft_consensus.cc:374] T 8b6a0f58c52f461bbddafa53ef95a98b P 12831c9e6cc0414fafe6350b67a3aba6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:20.395807 9908 raft_consensus.cc:729] T 8b6a0f58c52f461bbddafa53ef95a98b P 12831c9e6cc0414fafe6350b67a3aba6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12831c9e6cc0414fafe6350b67a3aba6, State: Initialized, Role: FOLLOWER
I20260807 05:09:20.395876 9906 raft_consensus.cc:348] T 8b6a0f58c52f461bbddafa53ef95a98b P b6ac299e54b8400fa308a5a99c053333 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } } peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } }
I20260807 05:09:20.396121 9906 raft_consensus.cc:374] T 8b6a0f58c52f461bbddafa53ef95a98b P b6ac299e54b8400fa308a5a99c053333 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:20.396184 9906 raft_consensus.cc:729] T 8b6a0f58c52f461bbddafa53ef95a98b P b6ac299e54b8400fa308a5a99c053333 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ac299e54b8400fa308a5a99c053333, State: Initialized, Role: FOLLOWER
I20260807 05:09:20.396235 9908 consensus_queue.cc:260] T 8b6a0f58c52f461bbddafa53ef95a98b P 12831c9e6cc0414fafe6350b67a3aba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } } peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } }
I20260807 05:09:20.396298 9907 raft_consensus.cc:348] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } } peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } }
I20260807 05:09:20.396514 9907 raft_consensus.cc:374] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:20.396569 9907 raft_consensus.cc:729] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3086396ef63f464db53ccef657279217, State: Initialized, Role: FOLLOWER
I20260807 05:09:20.396618 9906 consensus_queue.cc:260] T 8b6a0f58c52f461bbddafa53ef95a98b P b6ac299e54b8400fa308a5a99c053333 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } } peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } }
I20260807 05:09:20.396929 9907 consensus_queue.cc:260] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } } peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } }
I20260807 05:09:20.397192 9908 ts_tablet_manager.cc:1428] T 8b6a0f58c52f461bbddafa53ef95a98b P 12831c9e6cc0414fafe6350b67a3aba6: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260807 05:09:20.397428 9741 heartbeater.cc:503] Master 127.8.16.126:35739 was elected leader, sending a full tablet report...
I20260807 05:09:20.397578 9906 ts_tablet_manager.cc:1428] T 8b6a0f58c52f461bbddafa53ef95a98b P b6ac299e54b8400fa308a5a99c053333: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 05:09:20.397776 9907 ts_tablet_manager.cc:1428] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 05:09:20.398062 9600 heartbeater.cc:503] Master 127.8.16.126:35739 was elected leader, sending a full tablet report...
I20260807 05:09:20.398180 9882 heartbeater.cc:503] Master 127.8.16.126:35739 was elected leader, sending a full tablet report...
I20260807 05:09:20.444211 9914 raft_consensus.cc:482] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:09:20.444397 9914 raft_consensus.cc:504] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } } peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } }
I20260807 05:09:20.445417 9914 leader_election.cc:288] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6ac299e54b8400fa308a5a99c053333 (127.8.16.66:32779), 12831c9e6cc0414fafe6350b67a3aba6 (127.8.16.67:38771)
I20260807 05:09:20.449357 9904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.445837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:34083 (local address 127.8.16.67:38771)
0807 05:09:20.445988 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:20.445992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:20.446183 (+ 191us) server_negotiation.cc:407] Connection header received
0807 05:09:20.446366 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:20.446370 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:20.446424 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:20.446502 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:20.446810 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.447326 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.448485 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.448864 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.448951 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.449011 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:20.449172 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 05:09:20.449240 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 05:09:20.449364 9902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.446343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:59961 (local address 127.8.16.66:32779)
0807 05:09:20.446456 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:20.446460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:20.446471 (+ 11us) server_negotiation.cc:407] Connection header received
0807 05:09:20.446511 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:20.446513 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:20.446551 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:20.446613 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:20.446902 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.447394 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.448490 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.448877 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.448967 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.449025 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:20.449183 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 05:09:20.449248 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 05:09:20.449385 9915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.445683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:38771 (local address 127.8.16.65:34083)
0807 05:09:20.446113 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:20.446124 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:20.446240 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:20.446526 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:20.446529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:20.446548 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:20.446668 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.446676 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.447427 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.447430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.448374 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.448383 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.448968 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.448972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.449061 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.449079 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 05:09:20.449149 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 05:09:20.449219 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":18,"thread_start_us":98,"threads_started":1}
I20260807 05:09:20.449389 9916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.445671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:32779 (local address 127.8.16.65:59961)
0807 05:09:20.445998 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:20.446011 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:20.446123 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:20.446660 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:20.446663 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:20.446681 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:20.446778 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.446784 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.447496 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.447499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.448380 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.448389 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.448982 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.448985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.449082 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.449099 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 05:09:20.449159 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 05:09:20.449234 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":29,"thread_start_us":113,"threads_started":1}
I20260807 05:09:20.450129 9695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b6a0f58c52f461bbddafa53ef95a98b" candidate_uuid: "3086396ef63f464db53ccef657279217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ac299e54b8400fa308a5a99c053333" is_pre_election: true
I20260807 05:09:20.450133 9836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b6a0f58c52f461bbddafa53ef95a98b" candidate_uuid: "3086396ef63f464db53ccef657279217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12831c9e6cc0414fafe6350b67a3aba6" is_pre_election: true
I20260807 05:09:20.450385 9695 raft_consensus.cc:2454] T 8b6a0f58c52f461bbddafa53ef95a98b P b6ac299e54b8400fa308a5a99c053333 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3086396ef63f464db53ccef657279217 in term 0.
I20260807 05:09:20.450392 9836 raft_consensus.cc:2454] T 8b6a0f58c52f461bbddafa53ef95a98b P 12831c9e6cc0414fafe6350b67a3aba6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3086396ef63f464db53ccef657279217 in term 0.
I20260807 05:09:20.450810 9482 leader_election.cc:302] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12831c9e6cc0414fafe6350b67a3aba6, 3086396ef63f464db53ccef657279217; no voters:
I20260807 05:09:20.451046 9914 raft_consensus.cc:2781] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:09:20.451130 9914 raft_consensus.cc:482] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:09:20.451181 9914 raft_consensus.cc:3037] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:20.452173 9914 raft_consensus.cc:504] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } } peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } }
I20260807 05:09:20.452452 9914 leader_election.cc:288] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217 [CANDIDATE]: Term 1 election: Requested vote from peers b6ac299e54b8400fa308a5a99c053333 (127.8.16.66:32779), 12831c9e6cc0414fafe6350b67a3aba6 (127.8.16.67:38771)
I20260807 05:09:20.452746 9836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b6a0f58c52f461bbddafa53ef95a98b" candidate_uuid: "3086396ef63f464db53ccef657279217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12831c9e6cc0414fafe6350b67a3aba6"
I20260807 05:09:20.452745 9695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b6a0f58c52f461bbddafa53ef95a98b" candidate_uuid: "3086396ef63f464db53ccef657279217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ac299e54b8400fa308a5a99c053333"
I20260807 05:09:20.452840 9695 raft_consensus.cc:3037] T 8b6a0f58c52f461bbddafa53ef95a98b P b6ac299e54b8400fa308a5a99c053333 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:20.452848 9836 raft_consensus.cc:3037] T 8b6a0f58c52f461bbddafa53ef95a98b P 12831c9e6cc0414fafe6350b67a3aba6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:20.453879 9836 raft_consensus.cc:2454] T 8b6a0f58c52f461bbddafa53ef95a98b P 12831c9e6cc0414fafe6350b67a3aba6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3086396ef63f464db53ccef657279217 in term 1.
I20260807 05:09:20.454144 9482 leader_election.cc:302] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12831c9e6cc0414fafe6350b67a3aba6, 3086396ef63f464db53ccef657279217; no voters:
I20260807 05:09:20.454305 9914 raft_consensus.cc:2781] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:20.454344 9695 raft_consensus.cc:2454] T 8b6a0f58c52f461bbddafa53ef95a98b P b6ac299e54b8400fa308a5a99c053333 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3086396ef63f464db53ccef657279217 in term 1.
I20260807 05:09:20.454492 9914 raft_consensus.cc:686] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217 [term 1 LEADER]: Becoming Leader. State: Replica: 3086396ef63f464db53ccef657279217, State: Running, Role: LEADER
I20260807 05:09:20.454815 9914 consensus_queue.cc:237] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } } peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } }
I20260807 05:09:20.457718 9391 catalog_manager.cc:5495] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3086396ef63f464db53ccef657279217 (127.8.16.65). New cstate: current_term: 1 leader_uuid: "3086396ef63f464db53ccef657279217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } health_report { overall_health: HEALTHY } } }
I20260807 05:09:20.476943 9905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.473551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33638 (local address 127.8.16.65:34607)
0807 05:09:20.473683 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:20.473687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:20.473700 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:09:20.473880 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:20.473882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:20.473933 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:20.474012 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:20.474277 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.474763 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.475612 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.475822 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.475906 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.476222 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:20.476565 (+ 343us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:20.476670 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:20.476765 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 05:09:20.476818 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 05:09:20.482887 9836 raft_consensus.cc:1261] T 8b6a0f58c52f461bbddafa53ef95a98b P 12831c9e6cc0414fafe6350b67a3aba6 [term 1 FOLLOWER]: Refusing update from remote peer 3086396ef63f464db53ccef657279217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:20.482992 9695 raft_consensus.cc:1261] T 8b6a0f58c52f461bbddafa53ef95a98b P b6ac299e54b8400fa308a5a99c053333 [term 1 FOLLOWER]: Refusing update from remote peer 3086396ef63f464db53ccef657279217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:20.483629 9917 consensus_queue.cc:1035] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:20.483810 9914 consensus_queue.cc:1035] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:20.490940 9922 mvcc.cc:205] Tried to move back new op lower bound from 7315781060528132096 to 7315781060425445376. Current Snapshot: MvccSnapshot[applied={T|T < 7315781060528132096}]
I20260807 05:09:20.501472 9392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:20.501706 9392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:20.503985 9392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc: Sending DeleteTablet for 3 replicas of tablet 8b6a0f58c52f461bbddafa53ef95a98b
I20260807 05:09:20.504729 9816 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b6a0f58c52f461bbddafa53ef95a98b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:20 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:42916
I20260807 05:09:20.504741 9675 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b6a0f58c52f461bbddafa53ef95a98b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:20 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:39338
I20260807 05:09:20.504841 9531 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b6a0f58c52f461bbddafa53ef95a98b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:20 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:33624
I20260807 05:09:20.505268 9930 tablet_replica.cc:330] T 8b6a0f58c52f461bbddafa53ef95a98b P 12831c9e6cc0414fafe6350b67a3aba6: stopping tablet replica
I20260807 05:09:20.505529 9930 raft_consensus.cc:2229] T 8b6a0f58c52f461bbddafa53ef95a98b P 12831c9e6cc0414fafe6350b67a3aba6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:20.505766 9930 raft_consensus.cc:2258] T 8b6a0f58c52f461bbddafa53ef95a98b P 12831c9e6cc0414fafe6350b67a3aba6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:20.506227 9932 tablet_replica.cc:330] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217: stopping tablet replica
I20260807 05:09:20.506420 9932 raft_consensus.cc:2229] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:09:20.506614 9931 tablet_replica.cc:330] T 8b6a0f58c52f461bbddafa53ef95a98b P b6ac299e54b8400fa308a5a99c053333: stopping tablet replica
I20260807 05:09:20.506723 9932 raft_consensus.cc:2258] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:20.506986 9931 raft_consensus.cc:2229] T 8b6a0f58c52f461bbddafa53ef95a98b P b6ac299e54b8400fa308a5a99c053333 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:20.507325 9931 raft_consensus.cc:2258] T 8b6a0f58c52f461bbddafa53ef95a98b P b6ac299e54b8400fa308a5a99c053333 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:20.507460 9392 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:09:20.507865 9392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:09:20.508188 9932 ts_tablet_manager.cc:1905] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:20.509033 9931 ts_tablet_manager.cc:1905] T 8b6a0f58c52f461bbddafa53ef95a98b P b6ac299e54b8400fa308a5a99c053333: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:20.511020 9932 ts_tablet_manager.cc:1918] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:09:20.511139 9932 log.cc:1196] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-0/wal/wals/8b6a0f58c52f461bbddafa53ef95a98b
I20260807 05:09:20.511456 9932 ts_tablet_manager.cc:1939] T 8b6a0f58c52f461bbddafa53ef95a98b P 3086396ef63f464db53ccef657279217: Deleting consensus metadata
I20260807 05:09:20.511847 9931 ts_tablet_manager.cc:1918] T 8b6a0f58c52f461bbddafa53ef95a98b P b6ac299e54b8400fa308a5a99c053333: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:09:20.511993 9931 log.cc:1196] T 8b6a0f58c52f461bbddafa53ef95a98b P b6ac299e54b8400fa308a5a99c053333: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-1/wal/wals/8b6a0f58c52f461bbddafa53ef95a98b
I20260807 05:09:20.512346 9931 ts_tablet_manager.cc:1939] T 8b6a0f58c52f461bbddafa53ef95a98b P b6ac299e54b8400fa308a5a99c053333: Deleting consensus metadata
I20260807 05:09:20.512848 9675 tablet_service.cc:1467] Processing CreateTablet for tablet 90099bce7070417c81e5d5f34142aeb7 (DEFAULT_TABLE table=test-table [id=8c0455978ab7443a8079d84823973536]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:20.512897 9816 tablet_service.cc:1467] Processing CreateTablet for tablet 90099bce7070417c81e5d5f34142aeb7 (DEFAULT_TABLE table=test-table [id=8c0455978ab7443a8079d84823973536]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:20.513115 9816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90099bce7070417c81e5d5f34142aeb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:20.513115 9675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90099bce7070417c81e5d5f34142aeb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:20.515148 9906 tablet_bootstrap.cc:492] T 90099bce7070417c81e5d5f34142aeb7 P b6ac299e54b8400fa308a5a99c053333: Bootstrap starting.
I20260807 05:09:20.515339 9908 tablet_bootstrap.cc:492] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6: Bootstrap starting.
I20260807 05:09:20.515326 9380 catalog_manager.cc:4841] TS b6ac299e54b8400fa308a5a99c053333 (127.8.16.66:32779): tablet 8b6a0f58c52f461bbddafa53ef95a98b (table test-table [id=312f6893df1e4f35a7c56d5be70ea7e7]) successfully deleted
I20260807 05:09:20.515769 9379 catalog_manager.cc:4841] TS 3086396ef63f464db53ccef657279217 (127.8.16.65:34607): tablet 8b6a0f58c52f461bbddafa53ef95a98b (table test-table [id=312f6893df1e4f35a7c56d5be70ea7e7]) successfully deleted
I20260807 05:09:20.516065 9906 tablet_bootstrap.cc:654] T 90099bce7070417c81e5d5f34142aeb7 P b6ac299e54b8400fa308a5a99c053333: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:20.516278 9908 tablet_bootstrap.cc:654] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:20.516230 9531 tablet_service.cc:1467] Processing CreateTablet for tablet 90099bce7070417c81e5d5f34142aeb7 (DEFAULT_TABLE table=test-table [id=8c0455978ab7443a8079d84823973536]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:20.516456 9531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90099bce7070417c81e5d5f34142aeb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:20.517788 9906 tablet_bootstrap.cc:492] T 90099bce7070417c81e5d5f34142aeb7 P b6ac299e54b8400fa308a5a99c053333: No bootstrap required, opened a new log
I20260807 05:09:20.517866 9906 ts_tablet_manager.cc:1397] T 90099bce7070417c81e5d5f34142aeb7 P b6ac299e54b8400fa308a5a99c053333: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 05:09:20.518281 9906 raft_consensus.cc:348] T 90099bce7070417c81e5d5f34142aeb7 P b6ac299e54b8400fa308a5a99c053333 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } }
I20260807 05:09:20.518378 9906 raft_consensus.cc:374] T 90099bce7070417c81e5d5f34142aeb7 P b6ac299e54b8400fa308a5a99c053333 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:20.518404 9906 raft_consensus.cc:729] T 90099bce7070417c81e5d5f34142aeb7 P b6ac299e54b8400fa308a5a99c053333 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ac299e54b8400fa308a5a99c053333, State: Initialized, Role: FOLLOWER
I20260807 05:09:20.518522 9906 consensus_queue.cc:260] T 90099bce7070417c81e5d5f34142aeb7 P b6ac299e54b8400fa308a5a99c053333 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } }
I20260807 05:09:20.518591 9907 tablet_bootstrap.cc:492] T 90099bce7070417c81e5d5f34142aeb7 P 3086396ef63f464db53ccef657279217: Bootstrap starting.
I20260807 05:09:20.518755 9906 ts_tablet_manager.cc:1428] T 90099bce7070417c81e5d5f34142aeb7 P b6ac299e54b8400fa308a5a99c053333: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 05:09:20.519105 9930 ts_tablet_manager.cc:1905] T 8b6a0f58c52f461bbddafa53ef95a98b P 12831c9e6cc0414fafe6350b67a3aba6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:20.519495 9907 tablet_bootstrap.cc:654] T 90099bce7070417c81e5d5f34142aeb7 P 3086396ef63f464db53ccef657279217: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:20.520354 9908 tablet_bootstrap.cc:492] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6: No bootstrap required, opened a new log
I20260807 05:09:20.520444 9908 ts_tablet_manager.cc:1397] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 05:09:20.520867 9907 tablet_bootstrap.cc:492] T 90099bce7070417c81e5d5f34142aeb7 P 3086396ef63f464db53ccef657279217: No bootstrap required, opened a new log
I20260807 05:09:20.520957 9907 ts_tablet_manager.cc:1397] T 90099bce7070417c81e5d5f34142aeb7 P 3086396ef63f464db53ccef657279217: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:20.520901 9908 raft_consensus.cc:348] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } }
I20260807 05:09:20.521001 9908 raft_consensus.cc:374] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:20.521039 9908 raft_consensus.cc:729] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12831c9e6cc0414fafe6350b67a3aba6, State: Initialized, Role: FOLLOWER
I20260807 05:09:20.521160 9908 consensus_queue.cc:260] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } }
I20260807 05:09:20.521406 9908 ts_tablet_manager.cc:1428] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:09:20.521396 9907 raft_consensus.cc:348] T 90099bce7070417c81e5d5f34142aeb7 P 3086396ef63f464db53ccef657279217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } }
I20260807 05:09:20.521534 9907 raft_consensus.cc:374] T 90099bce7070417c81e5d5f34142aeb7 P 3086396ef63f464db53ccef657279217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:20.521579 9907 raft_consensus.cc:729] T 90099bce7070417c81e5d5f34142aeb7 P 3086396ef63f464db53ccef657279217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3086396ef63f464db53ccef657279217, State: Initialized, Role: FOLLOWER
I20260807 05:09:20.521694 9907 consensus_queue.cc:260] T 90099bce7070417c81e5d5f34142aeb7 P 3086396ef63f464db53ccef657279217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } }
I20260807 05:09:20.521938 9907 ts_tablet_manager.cc:1428] T 90099bce7070417c81e5d5f34142aeb7 P 3086396ef63f464db53ccef657279217: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:09:20.524205 9930 ts_tablet_manager.cc:1918] T 8b6a0f58c52f461bbddafa53ef95a98b P 12831c9e6cc0414fafe6350b67a3aba6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:09:20.524308 9930 log.cc:1196] T 8b6a0f58c52f461bbddafa53ef95a98b P 12831c9e6cc0414fafe6350b67a3aba6: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/8b6a0f58c52f461bbddafa53ef95a98b
I20260807 05:09:20.524562 9930 ts_tablet_manager.cc:1939] T 8b6a0f58c52f461bbddafa53ef95a98b P 12831c9e6cc0414fafe6350b67a3aba6: Deleting consensus metadata
I20260807 05:09:20.525578 9377 catalog_manager.cc:4841] TS 12831c9e6cc0414fafe6350b67a3aba6 (127.8.16.67:38771): tablet 8b6a0f58c52f461bbddafa53ef95a98b (table test-table [id=312f6893df1e4f35a7c56d5be70ea7e7]) successfully deleted
I20260807 05:09:20.536935 9912 raft_consensus.cc:482] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:09:20.537065 9912 raft_consensus.cc:504] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } }
I20260807 05:09:20.538259 9912 leader_election.cc:288] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3086396ef63f464db53ccef657279217 (127.8.16.65:34607), b6ac299e54b8400fa308a5a99c053333 (127.8.16.66:32779)
I20260807 05:09:20.542531 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.538452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:34607 (local address 127.8.16.67:55499)
0807 05:09:20.538726 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:20.538741 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:20.538812 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:20.539219 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:20.539221 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:20.539235 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:20.539356 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.539362 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.540087 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.540090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.541241 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.541252 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.542152 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.542155 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.542252 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.542276 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 05:09:20.542342 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 05:09:20.542418 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201}
I20260807 05:09:20.542729 9902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.538590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:38953 (local address 127.8.16.66:32779)
0807 05:09:20.538719 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:20.538723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:20.539091 (+ 368us) server_negotiation.cc:407] Connection header received
0807 05:09:20.539185 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:20.539188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:20.539232 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:20.539297 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:20.539570 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.540350 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.541728 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.542161 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.542256 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.542314 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:20.542528 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 05:09:20.542607 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 05:09:20.543105 9936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.538463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:32779 (local address 127.8.16.67:38953)
0807 05:09:20.538992 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:20.539005 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:20.539107 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:20.539312 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:20.539315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:20.539333 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:20.539452 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.539458 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.540468 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.540471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.541616 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.541624 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.542297 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.542301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.542409 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.542431 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 05:09:20.542501 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 05:09:20.542571 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":22,"thread_start_us":85,"threads_started":1}
I20260807 05:09:20.543151 9905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.538825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:55499 (local address 127.8.16.65:34607)
0807 05:09:20.538956 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:20.538959 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:20.538970 (+ 11us) server_negotiation.cc:407] Connection header received
0807 05:09:20.539022 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:20.539024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:20.539059 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:20.539144 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:20.539447 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.539987 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.541370 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.542036 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.542140 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.542206 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:20.542954 (+ 748us) server_negotiation.cc:299] Negotiation successful
0807 05:09:20.543037 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 05:09:20.543622 9543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90099bce7070417c81e5d5f34142aeb7" candidate_uuid: "12831c9e6cc0414fafe6350b67a3aba6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3086396ef63f464db53ccef657279217" is_pre_election: true
I20260807 05:09:20.543707 9695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90099bce7070417c81e5d5f34142aeb7" candidate_uuid: "12831c9e6cc0414fafe6350b67a3aba6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ac299e54b8400fa308a5a99c053333" is_pre_election: true
I20260807 05:09:20.543839 9695 raft_consensus.cc:2454] T 90099bce7070417c81e5d5f34142aeb7 P b6ac299e54b8400fa308a5a99c053333 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12831c9e6cc0414fafe6350b67a3aba6 in term 0.
I20260807 05:09:20.543855 9543 raft_consensus.cc:2454] T 90099bce7070417c81e5d5f34142aeb7 P 3086396ef63f464db53ccef657279217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12831c9e6cc0414fafe6350b67a3aba6 in term 0.
I20260807 05:09:20.544237 9768 leader_election.cc:302] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 12831c9e6cc0414fafe6350b67a3aba6, 3086396ef63f464db53ccef657279217, b6ac299e54b8400fa308a5a99c053333; no voters:
I20260807 05:09:20.544474 9912 raft_consensus.cc:2781] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:09:20.544535 9912 raft_consensus.cc:482] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:09:20.544577 9912 raft_consensus.cc:3037] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:20.545521 9912 raft_consensus.cc:504] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } }
I20260807 05:09:20.545866 9912 leader_election.cc:288] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6 [CANDIDATE]: Term 1 election: Requested vote from peers 3086396ef63f464db53ccef657279217 (127.8.16.65:34607), b6ac299e54b8400fa308a5a99c053333 (127.8.16.66:32779)
I20260807 05:09:20.546151 9543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90099bce7070417c81e5d5f34142aeb7" candidate_uuid: "12831c9e6cc0414fafe6350b67a3aba6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3086396ef63f464db53ccef657279217"
I20260807 05:09:20.546288 9543 raft_consensus.cc:3037] T 90099bce7070417c81e5d5f34142aeb7 P 3086396ef63f464db53ccef657279217 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:20.546276 9695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90099bce7070417c81e5d5f34142aeb7" candidate_uuid: "12831c9e6cc0414fafe6350b67a3aba6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ac299e54b8400fa308a5a99c053333"
I20260807 05:09:20.546381 9695 raft_consensus.cc:3037] T 90099bce7070417c81e5d5f34142aeb7 P b6ac299e54b8400fa308a5a99c053333 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:20.547418 9695 raft_consensus.cc:2454] T 90099bce7070417c81e5d5f34142aeb7 P b6ac299e54b8400fa308a5a99c053333 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12831c9e6cc0414fafe6350b67a3aba6 in term 1.
I20260807 05:09:20.547451 9543 raft_consensus.cc:2454] T 90099bce7070417c81e5d5f34142aeb7 P 3086396ef63f464db53ccef657279217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12831c9e6cc0414fafe6350b67a3aba6 in term 1.
I20260807 05:09:20.547689 9768 leader_election.cc:302] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12831c9e6cc0414fafe6350b67a3aba6, 3086396ef63f464db53ccef657279217; no voters:
I20260807 05:09:20.547920 9912 raft_consensus.cc:2781] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:20.548115 9912 raft_consensus.cc:686] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6 [term 1 LEADER]: Becoming Leader. State: Replica: 12831c9e6cc0414fafe6350b67a3aba6, State: Running, Role: LEADER
I20260807 05:09:20.548362 9912 consensus_queue.cc:237] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } }
I20260807 05:09:20.550486 9393 catalog_manager.cc:5495] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12831c9e6cc0414fafe6350b67a3aba6 (127.8.16.67). New cstate: current_term: 1 leader_uuid: "12831c9e6cc0414fafe6350b67a3aba6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } health_report { overall_health: HEALTHY } } }
W20260807 05:09:20.569895 9883 tablet.cc:2369] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 05:09:20 dist-test-slave-49h3 krb5kdc[9348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079359, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:20.588192 9444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.579412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60518 (local address 127.8.16.126:35739)
0807 05:09:20.579540 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:20.579544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:20.579715 (+ 171us) server_negotiation.cc:407] Connection header received
0807 05:09:20.579851 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:20.579853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:20.579894 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:20.579953 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:20.580716 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.581226 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.582367 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.582577 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.582666 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.585340 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:20.585343 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:20.585344 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:20.585365 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:20.586637 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:20.587052 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:20.587055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:20.587057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:20.587128 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:20.587358 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:20.587362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:20.587364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:20.587529 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:20.587622 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:20.587886 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 05:09:20.588039 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":48}
I20260807 05:09:20.588192 9944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.579336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:35739 (local address 127.0.0.1:60518)
0807 05:09:20.579620 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:20.579634 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:20.579760 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:20.579995 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:20.579998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:20.580490 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:20.580608 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.580616 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.581317 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.581320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.582238 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.582249 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.582700 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.582704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.582845 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.583502 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:20.583515 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:20.585204 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:20.586731 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:20.586740 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:20.586746 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:20.586964 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:20.587204 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:20.587207 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:20.587209 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:20.587286 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:20.587623 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:20.587630 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:20.587737 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 05:09:20.587830 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 05:09:20.587988 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":42,"thread_start_us":101,"threads_started":1}
I20260807 05:09:20.592074 9393 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:60518:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 05:09:20.592774 9393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:09:20.597574 9531 tablet_service.cc:1467] Processing CreateTablet for tablet 05cde427393242369615498ed4f1430d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e94a729914714288933ea72d7d85b85c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 05:09:20.597905 9531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05cde427393242369615498ed4f1430d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:20.597851 9816 tablet_service.cc:1467] Processing CreateTablet for tablet 05cde427393242369615498ed4f1430d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e94a729914714288933ea72d7d85b85c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 05:09:20.598214 9816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05cde427393242369615498ed4f1430d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:20.600199 9907 tablet_bootstrap.cc:492] T 05cde427393242369615498ed4f1430d P 3086396ef63f464db53ccef657279217: Bootstrap starting.
I20260807 05:09:20.600500 9675 tablet_service.cc:1467] Processing CreateTablet for tablet 05cde427393242369615498ed4f1430d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e94a729914714288933ea72d7d85b85c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 05:09:20.600849 9675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05cde427393242369615498ed4f1430d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:20.601197 9907 tablet_bootstrap.cc:654] T 05cde427393242369615498ed4f1430d P 3086396ef63f464db53ccef657279217: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:20.601758 9908 tablet_bootstrap.cc:492] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6: Bootstrap starting.
I20260807 05:09:20.602646 9907 tablet_bootstrap.cc:492] T 05cde427393242369615498ed4f1430d P 3086396ef63f464db53ccef657279217: No bootstrap required, opened a new log
I20260807 05:09:20.602641 9908 tablet_bootstrap.cc:654] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6: Neither blocks nor log segments found. Creating new log.
W20260807 05:09:20.602689 9742 tablet.cc:2369] T 90099bce7070417c81e5d5f34142aeb7 P b6ac299e54b8400fa308a5a99c053333: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:09:20.602722 9907 ts_tablet_manager.cc:1397] T 05cde427393242369615498ed4f1430d P 3086396ef63f464db53ccef657279217: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 05:09:20.603147 9907 raft_consensus.cc:348] T 05cde427393242369615498ed4f1430d P 3086396ef63f464db53ccef657279217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } }
I20260807 05:09:20.603277 9907 raft_consensus.cc:374] T 05cde427393242369615498ed4f1430d P 3086396ef63f464db53ccef657279217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:20.603308 9907 raft_consensus.cc:729] T 05cde427393242369615498ed4f1430d P 3086396ef63f464db53ccef657279217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3086396ef63f464db53ccef657279217, State: Initialized, Role: FOLLOWER
I20260807 05:09:20.603439 9907 consensus_queue.cc:260] T 05cde427393242369615498ed4f1430d P 3086396ef63f464db53ccef657279217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } }
I20260807 05:09:20.603626 9906 tablet_bootstrap.cc:492] T 05cde427393242369615498ed4f1430d P b6ac299e54b8400fa308a5a99c053333: Bootstrap starting.
I20260807 05:09:20.603718 9907 ts_tablet_manager.cc:1428] T 05cde427393242369615498ed4f1430d P 3086396ef63f464db53ccef657279217: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:09:20.603685 9925 tablet_replica.cc:439] T 05cde427393242369615498ed4f1430d P 3086396ef63f464db53ccef657279217: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } } }
I20260807 05:09:20.603816 9925 tablet_replica.cc:442] T 05cde427393242369615498ed4f1430d P 3086396ef63f464db53ccef657279217: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:09:20.604444 9906 tablet_bootstrap.cc:654] T 05cde427393242369615498ed4f1430d P b6ac299e54b8400fa308a5a99c053333: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:20.605623 9908 tablet_bootstrap.cc:492] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6: No bootstrap required, opened a new log
I20260807 05:09:20.605717 9908 ts_tablet_manager.cc:1397] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260807 05:09:20.605952 9906 tablet_bootstrap.cc:492] T 05cde427393242369615498ed4f1430d P b6ac299e54b8400fa308a5a99c053333: No bootstrap required, opened a new log
I20260807 05:09:20.606029 9906 ts_tablet_manager.cc:1397] T 05cde427393242369615498ed4f1430d P b6ac299e54b8400fa308a5a99c053333: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 05:09:20.606146 9908 raft_consensus.cc:348] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } }
I20260807 05:09:20.606285 9908 raft_consensus.cc:374] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:20.606315 9908 raft_consensus.cc:729] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12831c9e6cc0414fafe6350b67a3aba6, State: Initialized, Role: FOLLOWER
I20260807 05:09:20.606446 9908 consensus_queue.cc:260] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } }
I20260807 05:09:20.606467 9906 raft_consensus.cc:348] T 05cde427393242369615498ed4f1430d P b6ac299e54b8400fa308a5a99c053333 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } }
I20260807 05:09:20.606564 9906 raft_consensus.cc:374] T 05cde427393242369615498ed4f1430d P b6ac299e54b8400fa308a5a99c053333 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:20.606593 9906 raft_consensus.cc:729] T 05cde427393242369615498ed4f1430d P b6ac299e54b8400fa308a5a99c053333 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ac299e54b8400fa308a5a99c053333, State: Initialized, Role: FOLLOWER
I20260807 05:09:20.606727 9908 ts_tablet_manager.cc:1428] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 05:09:20.606699 9906 consensus_queue.cc:260] T 05cde427393242369615498ed4f1430d P b6ac299e54b8400fa308a5a99c053333 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } }
I20260807 05:09:20.606695 9912 tablet_replica.cc:439] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } } }
I20260807 05:09:20.606813 9912 tablet_replica.cc:442] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:09:20.606942 9906 ts_tablet_manager.cc:1428] T 05cde427393242369615498ed4f1430d P b6ac299e54b8400fa308a5a99c053333: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 05:09:20.607125 9913 tablet_replica.cc:439] T 05cde427393242369615498ed4f1430d P b6ac299e54b8400fa308a5a99c053333: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } } }
I20260807 05:09:20.607276 9913 tablet_replica.cc:442] T 05cde427393242369615498ed4f1430d P b6ac299e54b8400fa308a5a99c053333: This TxnStatusTablet replica's current role is: FOLLOWER
W20260807 05:09:20.643095 9601 tablet.cc:2369] T 05cde427393242369615498ed4f1430d P 3086396ef63f464db53ccef657279217: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:09:20.706559 9912 raft_consensus.cc:482] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:09:20.706709 9912 raft_consensus.cc:504] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } }
I20260807 05:09:20.707152 9912 leader_election.cc:288] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3086396ef63f464db53ccef657279217 (127.8.16.65:34607), b6ac299e54b8400fa308a5a99c053333 (127.8.16.66:32779)
I20260807 05:09:20.707659 9695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05cde427393242369615498ed4f1430d" candidate_uuid: "12831c9e6cc0414fafe6350b67a3aba6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ac299e54b8400fa308a5a99c053333" is_pre_election: true
I20260807 05:09:20.707659 9543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05cde427393242369615498ed4f1430d" candidate_uuid: "12831c9e6cc0414fafe6350b67a3aba6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3086396ef63f464db53ccef657279217" is_pre_election: true
I20260807 05:09:20.707820 9695 raft_consensus.cc:2454] T 05cde427393242369615498ed4f1430d P b6ac299e54b8400fa308a5a99c053333 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12831c9e6cc0414fafe6350b67a3aba6 in term 0.
I20260807 05:09:20.707820 9543 raft_consensus.cc:2454] T 05cde427393242369615498ed4f1430d P 3086396ef63f464db53ccef657279217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12831c9e6cc0414fafe6350b67a3aba6 in term 0.
I20260807 05:09:20.708127 9769 leader_election.cc:302] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12831c9e6cc0414fafe6350b67a3aba6, b6ac299e54b8400fa308a5a99c053333; no voters:
I20260807 05:09:20.708345 9912 raft_consensus.cc:2781] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:09:20.708407 9912 raft_consensus.cc:482] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:09:20.708437 9912 raft_consensus.cc:3037] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:20.709483 9912 raft_consensus.cc:504] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } }
I20260807 05:09:20.709790 9912 leader_election.cc:288] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6 [CANDIDATE]: Term 1 election: Requested vote from peers 3086396ef63f464db53ccef657279217 (127.8.16.65:34607), b6ac299e54b8400fa308a5a99c053333 (127.8.16.66:32779)
I20260807 05:09:20.710161 9543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05cde427393242369615498ed4f1430d" candidate_uuid: "12831c9e6cc0414fafe6350b67a3aba6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3086396ef63f464db53ccef657279217"
I20260807 05:09:20.710179 9695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05cde427393242369615498ed4f1430d" candidate_uuid: "12831c9e6cc0414fafe6350b67a3aba6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ac299e54b8400fa308a5a99c053333"
I20260807 05:09:20.710281 9543 raft_consensus.cc:3037] T 05cde427393242369615498ed4f1430d P 3086396ef63f464db53ccef657279217 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:20.710291 9695 raft_consensus.cc:3037] T 05cde427393242369615498ed4f1430d P b6ac299e54b8400fa308a5a99c053333 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:20.711177 9543 raft_consensus.cc:2454] T 05cde427393242369615498ed4f1430d P 3086396ef63f464db53ccef657279217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12831c9e6cc0414fafe6350b67a3aba6 in term 1.
I20260807 05:09:20.711177 9695 raft_consensus.cc:2454] T 05cde427393242369615498ed4f1430d P b6ac299e54b8400fa308a5a99c053333 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12831c9e6cc0414fafe6350b67a3aba6 in term 1.
I20260807 05:09:20.711454 9769 leader_election.cc:302] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12831c9e6cc0414fafe6350b67a3aba6, b6ac299e54b8400fa308a5a99c053333; no voters:
I20260807 05:09:20.711606 9912 raft_consensus.cc:2781] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:20.711690 9912 raft_consensus.cc:686] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6 [term 1 LEADER]: Becoming Leader. State: Replica: 12831c9e6cc0414fafe6350b67a3aba6, State: Running, Role: LEADER
I20260807 05:09:20.711820 9912 consensus_queue.cc:237] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } }
I20260807 05:09:20.712473 9937 tablet_replica.cc:439] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6: TxnStatusTablet state changed. Reason: New leader 12831c9e6cc0414fafe6350b67a3aba6. Latest consensus state: current_term: 1 leader_uuid: "12831c9e6cc0414fafe6350b67a3aba6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } } }
I20260807 05:09:20.712577 9937 tablet_replica.cc:442] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6: This TxnStatusTablet replica's current role is: LEADER
I20260807 05:09:20.712872 9949 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 05:09:20.713040 9949 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 05:09:20.713567 9392 catalog_manager.cc:5495] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12831c9e6cc0414fafe6350b67a3aba6 (127.8.16.67). New cstate: current_term: 1 leader_uuid: "12831c9e6cc0414fafe6350b67a3aba6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } health_report { overall_health: HEALTHY } } }
I20260807 05:09:20.734650 9944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.730689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:38771 (local address 127.0.0.1:42918)
0807 05:09:20.730827 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:20.730840 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:20.730946 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:20.731254 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:20.731257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:20.731265 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 05:09:20.731384 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.731390 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.732115 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.732118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.732778 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.732789 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.733343 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.733346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.733426 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.733509 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 05:09:20.734375 (+ 866us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 05:09:20.734389 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 05:09:20.734446 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 05:09:20.734524 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 05:09:20.734676 9904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.730815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42918 (local address 127.8.16.67:38771)
0807 05:09:20.730947 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:20.730952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:20.730966 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:09:20.731082 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:20.731085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:20.731153 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:20.731246 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:20.731486 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.732002 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.732884 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.733161 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.733260 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.733657 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:20.734261 (+ 604us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:20.734400 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:20.734462 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 05:09:20.734524 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 05:09:20.737849 9695 raft_consensus.cc:1261] T 05cde427393242369615498ed4f1430d P b6ac299e54b8400fa308a5a99c053333 [term 1 FOLLOWER]: Refusing update from remote peer 12831c9e6cc0414fafe6350b67a3aba6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:20.737994 9543 raft_consensus.cc:1261] T 05cde427393242369615498ed4f1430d P 3086396ef63f464db53ccef657279217 [term 1 FOLLOWER]: Refusing update from remote peer 12831c9e6cc0414fafe6350b67a3aba6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:20.738339 9937 consensus_queue.cc:1035] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:20.738899 9937 consensus_queue.cc:1035] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:20.739842 9913 tablet_replica.cc:439] T 05cde427393242369615498ed4f1430d P b6ac299e54b8400fa308a5a99c053333: TxnStatusTablet state changed. Reason: New leader 12831c9e6cc0414fafe6350b67a3aba6. Latest consensus state: current_term: 1 leader_uuid: "12831c9e6cc0414fafe6350b67a3aba6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } } }
I20260807 05:09:20.739956 9913 tablet_replica.cc:442] T 05cde427393242369615498ed4f1430d P b6ac299e54b8400fa308a5a99c053333: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:09:20.740453 9925 tablet_replica.cc:439] T 05cde427393242369615498ed4f1430d P 3086396ef63f464db53ccef657279217: TxnStatusTablet state changed. Reason: New leader 12831c9e6cc0414fafe6350b67a3aba6. Latest consensus state: current_term: 1 leader_uuid: "12831c9e6cc0414fafe6350b67a3aba6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } } }
I20260807 05:09:20.740563 9925 tablet_replica.cc:442] T 05cde427393242369615498ed4f1430d P 3086396ef63f464db53ccef657279217: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:09:20.742743 9937 tablet_replica.cc:439] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "12831c9e6cc0414fafe6350b67a3aba6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } } }
I20260807 05:09:20.742847 9937 tablet_replica.cc:442] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6: This TxnStatusTablet replica's current role is: LEADER
I20260807 05:09:20.742960 9937 tablet_replica.cc:439] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "12831c9e6cc0414fafe6350b67a3aba6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } } }
I20260807 05:09:20.743044 9937 tablet_replica.cc:442] T 05cde427393242369615498ed4f1430d P 12831c9e6cc0414fafe6350b67a3aba6: This TxnStatusTablet replica's current role is: LEADER
I20260807 05:09:20.743234 9921 mvcc.cc:205] Tried to move back new op lower bound from 7315781061577261056 to 7315781061476880384. Current Snapshot: MvccSnapshot[applied={T|T < 7315781061577261056}]
I20260807 05:09:20.743755 9913 tablet_replica.cc:439] T 05cde427393242369615498ed4f1430d P b6ac299e54b8400fa308a5a99c053333: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "12831c9e6cc0414fafe6350b67a3aba6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } } }
I20260807 05:09:20.743858 9913 tablet_replica.cc:442] T 05cde427393242369615498ed4f1430d P b6ac299e54b8400fa308a5a99c053333: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:09:20.745909 9925 tablet_replica.cc:439] T 05cde427393242369615498ed4f1430d P 3086396ef63f464db53ccef657279217: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "12831c9e6cc0414fafe6350b67a3aba6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 } } peers { permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 } } peers { permanent_uuid: "12831c9e6cc0414fafe6350b67a3aba6" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38771 } } }
I20260807 05:09:20.746016 9925 tablet_replica.cc:442] T 05cde427393242369615498ed4f1430d P 3086396ef63f464db53ccef657279217: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:09:20.751803 9904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.748600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42928 (local address 127.8.16.67:38771)
0807 05:09:20.748710 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:20.748713 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:20.748845 (+ 132us) server_negotiation.cc:407] Connection header received
0807 05:09:20.748970 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:20.748972 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:20.749010 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:20.749067 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:20.749284 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.749842 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.750946 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.751158 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.751224 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.751456 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:20.751517 (+ 61us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:20.751560 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:20.751652 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 05:09:20.751695 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 05:09:20.762210 9904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.758816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:51549 (local address 127.8.16.67:38771)
0807 05:09:20.758902 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:20.758906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:20.758935 (+ 29us) server_negotiation.cc:407] Connection header received
0807 05:09:20.759063 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:20.759065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:20.759100 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:20.759147 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:20.759393 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.760023 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.761223 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.761423 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.761531 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.761796 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:20.761854 (+ 58us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:20.761901 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:20.761991 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 05:09:20.762059 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":21}
I20260807 05:09:20.762230 9888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.758736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:38771 (local address 127.8.16.67:51549)
0807 05:09:20.758879 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:20.758893 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:20.758982 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:20.759170 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:20.759173 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:20.759179 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 05:09:20.759311 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.759316 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.760127 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.760130 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.761106 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:20.761116 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.761534 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.761538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:20.761631 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.761689 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 05:09:20.761918 (+ 229us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 05:09:20.761923 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 05:09:20.761985 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 05:09:20.762095 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":38}
I20260807 05:09:20.768551 9695 raft_consensus.cc:1261] T 90099bce7070417c81e5d5f34142aeb7 P b6ac299e54b8400fa308a5a99c053333 [term 1 FOLLOWER]: Refusing update from remote peer 12831c9e6cc0414fafe6350b67a3aba6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:20.768581 9543 raft_consensus.cc:1261] T 90099bce7070417c81e5d5f34142aeb7 P 3086396ef63f464db53ccef657279217 [term 1 FOLLOWER]: Refusing update from remote peer 12831c9e6cc0414fafe6350b67a3aba6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:20.768966 9912 consensus_queue.cc:1035] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ac299e54b8400fa308a5a99c053333" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:20.769184 9937 consensus_queue.cc:1035] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3086396ef63f464db53ccef657279217" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:20.773159 9920 mvcc.cc:205] Tried to move back new op lower bound from 7315781061704884224 to 7315781060808458240. Current Snapshot: MvccSnapshot[applied={T|T < 7315781061704884224}]
I20260807 05:09:20.808815 9392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:20.808970 9392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:20.810896 9392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 13fd3c6b893b4878ac63c3f251b309fc: Sending DeleteTablet for 3 replicas of tablet 90099bce7070417c81e5d5f34142aeb7
I20260807 05:09:20.811436 9675 tablet_service.cc:1514] Processing DeleteTablet for tablet 90099bce7070417c81e5d5f34142aeb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:20 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:39338
I20260807 05:09:20.811429 9531 tablet_service.cc:1514] Processing DeleteTablet for tablet 90099bce7070417c81e5d5f34142aeb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:20 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:33624
I20260807 05:09:20.811528 9815 tablet_service.cc:1514] Processing DeleteTablet for tablet 90099bce7070417c81e5d5f34142aeb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:20 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:42916
I20260807 05:09:20.811607 9932 tablet_replica.cc:330] T 90099bce7070417c81e5d5f34142aeb7 P 3086396ef63f464db53ccef657279217: stopping tablet replica
I20260807 05:09:20.811625 9931 tablet_replica.cc:330] T 90099bce7070417c81e5d5f34142aeb7 P b6ac299e54b8400fa308a5a99c053333: stopping tablet replica
I20260807 05:09:20.811717 9932 raft_consensus.cc:2229] T 90099bce7070417c81e5d5f34142aeb7 P 3086396ef63f464db53ccef657279217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:20.811733 9931 raft_consensus.cc:2229] T 90099bce7070417c81e5d5f34142aeb7 P b6ac299e54b8400fa308a5a99c053333 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:20.811811 9930 tablet_replica.cc:330] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6: stopping tablet replica
I20260807 05:09:20.811834 9931 raft_consensus.cc:2258] T 90099bce7070417c81e5d5f34142aeb7 P b6ac299e54b8400fa308a5a99c053333 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:20.811830 9932 raft_consensus.cc:2258] T 90099bce7070417c81e5d5f34142aeb7 P 3086396ef63f464db53ccef657279217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:20.811918 9930 raft_consensus.cc:2229] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:09:20.812191 9930 raft_consensus.cc:2258] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:20.812582 9931 ts_tablet_manager.cc:1905] T 90099bce7070417c81e5d5f34142aeb7 P b6ac299e54b8400fa308a5a99c053333: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:20.812582 9932 ts_tablet_manager.cc:1905] T 90099bce7070417c81e5d5f34142aeb7 P 3086396ef63f464db53ccef657279217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:20.812873 9930 ts_tablet_manager.cc:1905] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:20.815403 9930 ts_tablet_manager.cc:1918] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:09:20.815403 9931 ts_tablet_manager.cc:1918] T 90099bce7070417c81e5d5f34142aeb7 P b6ac299e54b8400fa308a5a99c053333: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:09:20.815431 9932 ts_tablet_manager.cc:1918] T 90099bce7070417c81e5d5f34142aeb7 P 3086396ef63f464db53ccef657279217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:09:20.815487 9931 log.cc:1196] T 90099bce7070417c81e5d5f34142aeb7 P b6ac299e54b8400fa308a5a99c053333: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-1/wal/wals/90099bce7070417c81e5d5f34142aeb7
I20260807 05:09:20.815487 9930 log.cc:1196] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/90099bce7070417c81e5d5f34142aeb7
I20260807 05:09:20.815503 9932 log.cc:1196] T 90099bce7070417c81e5d5f34142aeb7 P 3086396ef63f464db53ccef657279217: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079354071073-8257-0/minicluster-data/ts-0/wal/wals/90099bce7070417c81e5d5f34142aeb7
I20260807 05:09:20.815753 9930 ts_tablet_manager.cc:1939] T 90099bce7070417c81e5d5f34142aeb7 P 12831c9e6cc0414fafe6350b67a3aba6: Deleting consensus metadata
I20260807 05:09:20.815761 9932 ts_tablet_manager.cc:1939] T 90099bce7070417c81e5d5f34142aeb7 P 3086396ef63f464db53ccef657279217: Deleting consensus metadata
I20260807 05:09:20.815757 9931 ts_tablet_manager.cc:1939] T 90099bce7070417c81e5d5f34142aeb7 P b6ac299e54b8400fa308a5a99c053333: Deleting consensus metadata
I20260807 05:09:20.816411 9379 catalog_manager.cc:4841] TS 3086396ef63f464db53ccef657279217 (127.8.16.65:34607): tablet 90099bce7070417c81e5d5f34142aeb7 (table test-table [id=8c0455978ab7443a8079d84823973536]) successfully deleted
I20260807 05:09:20.816411 9380 catalog_manager.cc:4841] TS b6ac299e54b8400fa308a5a99c053333 (127.8.16.66:32779): tablet 90099bce7070417c81e5d5f34142aeb7 (table test-table [id=8c0455978ab7443a8079d84823973536]) successfully deleted
I20260807 05:09:20.816411 9377 catalog_manager.cc:4841] TS 12831c9e6cc0414fafe6350b67a3aba6 (127.8.16.67:38771): tablet 90099bce7070417c81e5d5f34142aeb7 (table test-table [id=8c0455978ab7443a8079d84823973536]) successfully deleted
Aug 07 05:09:20 dist-test-slave-49h3 krb5kdc[9348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079360, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.16.65@KRBTEST.COM
I20260807 05:09:20.827210 9905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.817646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33652 (local address 127.8.16.65:34607)
0807 05:09:20.817771 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:20.817776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:20.817989 (+ 213us) server_negotiation.cc:407] Connection header received
0807 05:09:20.818101 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:20.818104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:20.818149 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:20.818222 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:20.818885 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.819394 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.820194 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.820414 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.820496 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.823836 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:20.823849 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:20.823863 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:20.823887 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:20.825484 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:20.825962 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:20.825967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:20.825968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:20.826050 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:20.826322 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:20.826324 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:20.826325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:20.826562 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:20.826722 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:20.826955 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 05:09:20.827058 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 05:09:20.827819 9594 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33652
I20260807 05:09:20.836345 9444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:20.829588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60532 (local address 127.8.16.126:35739)
0807 05:09:20.829738 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:20.829742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:20.829757 (+ 15us) server_negotiation.cc:407] Connection header received
0807 05:09:20.829833 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:20.829836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:20.829879 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:20.829939 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:20.830545 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.831061 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.831819 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:20.832063 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:20.832150 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:20.833431 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:20.833435 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:20.833438 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:20.833527 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:20.834885 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:20.835318 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:20.835320 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:20.835322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:20.835388 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:20.835628 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:20.835630 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:20.835631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:20.835798 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:20.835891 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:20.836119 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 05:09:20.836214 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260807 05:09:20.836838 9392 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60532
I20260807 05:09:20.838174 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 9466
I20260807 05:09:20.848356 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 9609
I20260807 05:09:20.854628 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 9750
I20260807 05:09:20.860371 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 9361
2026-08-07T05:09:20Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2535 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:09:20 dist-test-slave-49h3 krb5kdc[9966](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:09:20 dist-test-slave-49h3 krb5kdc[9966](info): set up 2 sockets
Aug 07 05:09:20 dist-test-slave-49h3 krb5kdc[9966](info): commencing operation
krb5kdc: starting...
W20260807 05:09:21.837078 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.955s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:09:21 dist-test-slave-49h3 krb5kdc[9966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:09:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:09:21.956663 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:35103
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:43529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:35103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:22.064244 9979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:22.064524 9979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:22.064566 9979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:22.068614 9979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:22.068686 9979 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 05:09:22.068702 9979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:22.068719 9979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:22.068734 9979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:22.068753 9979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:22.074110 9979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:43529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:35103
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:35103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:22.075213 9979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:22.076275 9979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:22.084591 9987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:22.084584 9984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:22.084657 9979 server_base.cc:1034] running on GCE node
W20260807 05:09:22.084584 9985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:22.085404 9979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:22.086556 9979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:22.087713 9979 hybrid_clock.cc:648] HybridClock initialized: now 1786079362087705 us; error 31 us; skew 500 ppm
Aug 07 05:09:22 dist-test-slave-49h3 krb5kdc[9966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079362, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:22.090430 9979 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:09:22.091341 9979 webserver.cc:460] Webserver started at http://127.8.16.126:37561/ using document root <none> and password file <none>
I20260807 05:09:22.091878 9979 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:22.091943 9979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:22.092133 9979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:22.094238 9979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "ab5ceab05697435281630d85fe06610e"
format_stamp: "Formatted at 2026-08-07 05:09:22 on dist-test-slave-49h3"
server_key: "26bbb93a5760c8dac6fd5a8ec347f9cb"
server_key_iv: "b926d0c6e3264a4544a9d1996e757f26"
server_key_version: "encryptionkey@0"
I20260807 05:09:22.094803 9979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "ab5ceab05697435281630d85fe06610e"
format_stamp: "Formatted at 2026-08-07 05:09:22 on dist-test-slave-49h3"
server_key: "26bbb93a5760c8dac6fd5a8ec347f9cb"
server_key_iv: "b926d0c6e3264a4544a9d1996e757f26"
server_key_version: "encryptionkey@0"
I20260807 05:09:22.098503 9979 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:09:22.100840 9994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:22.102108 9979 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:22.102217 9979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "ab5ceab05697435281630d85fe06610e"
format_stamp: "Formatted at 2026-08-07 05:09:22 on dist-test-slave-49h3"
server_key: "26bbb93a5760c8dac6fd5a8ec347f9cb"
server_key_iv: "b926d0c6e3264a4544a9d1996e757f26"
server_key_version: "encryptionkey@0"
I20260807 05:09:22.102325 9979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:22.123116 9979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:22.126039 9979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:22.126206 9979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:22.133617 9979 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:35103
I20260807 05:09:22.133648 10057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:35103 every 8 connection(s)
I20260807 05:09:22.134503 9979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:22.137244 10058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:22.142534 10058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e: Bootstrap starting.
I20260807 05:09:22.143133 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 9979
I20260807 05:09:22.143230 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:09:22.143472 8257 external_mini_cluster.cc:1428] Setting key 0c9193107d4ae2f0ecd770a4e96dd3e1
I20260807 05:09:22.145041 10058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:22.145774 10058 log.cc:824] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:22.147792 10058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e: No bootstrap required, opened a new log
I20260807 05:09:22.151513 10058 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5ceab05697435281630d85fe06610e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35103 } }
I20260807 05:09:22.151727 10058 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:22.151783 10058 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab5ceab05697435281630d85fe06610e, State: Initialized, Role: FOLLOWER
Aug 07 05:09:22 dist-test-slave-49h3 krb5kdc[9966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:22.152222 10058 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5ceab05697435281630d85fe06610e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35103 } }
I20260807 05:09:22.152380 10058 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:09:22.152447 10058 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:09:22.152544 10058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:22.153365 10058 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5ceab05697435281630d85fe06610e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35103 } }
I20260807 05:09:22.153733 10058 leader_election.cc:302] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab5ceab05697435281630d85fe06610e; no voters:
I20260807 05:09:22.154068 10058 leader_election.cc:288] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:09:22.154157 10063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:22.154381 10063 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e [term 1 LEADER]: Becoming Leader. State: Replica: ab5ceab05697435281630d85fe06610e, State: Running, Role: LEADER
I20260807 05:09:22.154709 10063 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5ceab05697435281630d85fe06610e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35103 } }
I20260807 05:09:22.155277 10058 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:22.156278 10064 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab5ceab05697435281630d85fe06610e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5ceab05697435281630d85fe06610e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35103 } } }
I20260807 05:09:22.156394 10064 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:22.156751 10065 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab5ceab05697435281630d85fe06610e. Latest consensus state: current_term: 1 leader_uuid: "ab5ceab05697435281630d85fe06610e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5ceab05697435281630d85fe06610e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35103 } } }
I20260807 05:09:22.156901 10065 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:22.157533 10072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:22.158434 10061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:22.144676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48662 (local address 127.8.16.126:35103)
0807 05:09:22.145071 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:22.145085 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:22.145118 (+ 33us) server_negotiation.cc:407] Connection header received
0807 05:09:22.145777 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:22.145792 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:22.146137 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:22.146404 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:22.147117 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.148536 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.149386 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.150097 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.150273 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:22.152638 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:22.152644 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:22.152660 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:22.152689 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:22.154624 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:22.155139 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:22.155145 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:22.155158 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:22.155300 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:22.155637 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:22.155640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:22.155642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:22.156129 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:22.157027 (+ 898us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:22.157198 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 05:09:22.157523 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":123,"threads_started":1}
I20260807 05:09:22.160233 10072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:22.165341 10072 catalog_manager.cc:1347] Generated new cluster ID: b04a544d0d004e2595f69560a303817b
I20260807 05:09:22.165406 10072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:22.186014 10072 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:09:22.186992 10072 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:22.192603 10072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e: Generated new TSK 0
I20260807 05:09:22.193243 10072 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
I20260807 05:09:22.221614 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:35103
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:43529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:22.330619 10084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:22.330861 10084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:22.330905 10084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:22.334807 10084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:22.334859 10084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:22.334897 10084 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 05:09:22.334915 10084 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 05:09:22.334945 10084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:09:22.340116 10084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:43529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.16.126:35103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:22.341061 10084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:22.342022 10084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:22.351222 10089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:22.351298 10084 server_base.cc:1034] running on GCE node
W20260807 05:09:22.351229 10090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:22.351199 10092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:22.351862 10084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:22.352458 10084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:22.353627 10084 hybrid_clock.cc:648] HybridClock initialized: now 1786079362353613 us; error 27 us; skew 500 ppm
Aug 07 05:09:22 dist-test-slave-49h3 krb5kdc[9966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079362, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:22.356372 10084 init.cc:377] Logged in from keytab as kudu/127.8.16.65@KRBTEST.COM (short username kudu)
I20260807 05:09:22.357266 10084 webserver.cc:460] Webserver started at http://127.8.16.65:46045/ using document root <none> and password file <none>
I20260807 05:09:22.357827 10084 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:22.357896 10084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:22.358083 10084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:22.360071 10084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "7985da867e914492888a84a9ae418d56"
format_stamp: "Formatted at 2026-08-07 05:09:22 on dist-test-slave-49h3"
server_key: "2e111361139665bc9fcba75a6fa810cf"
server_key_iv: "6e466372750c8361341227713760dd42"
server_key_version: "encryptionkey@0"
I20260807 05:09:22.360606 10084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "7985da867e914492888a84a9ae418d56"
format_stamp: "Formatted at 2026-08-07 05:09:22 on dist-test-slave-49h3"
server_key: "2e111361139665bc9fcba75a6fa810cf"
server_key_iv: "6e466372750c8361341227713760dd42"
server_key_version: "encryptionkey@0"
I20260807 05:09:22.364148 10084 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:09:22.366439 10099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:22.367543 10084 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:22.367650 10084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "7985da867e914492888a84a9ae418d56"
format_stamp: "Formatted at 2026-08-07 05:09:22 on dist-test-slave-49h3"
server_key: "2e111361139665bc9fcba75a6fa810cf"
server_key_iv: "6e466372750c8361341227713760dd42"
server_key_version: "encryptionkey@0"
I20260807 05:09:22.367738 10084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:22.386514 10084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:22.389729 10084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:22.389922 10084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:22.391360 10084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:22.391412 10084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:22.391464 10084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:22.391491 10084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 05:09:22 dist-test-slave-49h3 krb5kdc[9966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079362, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:22.401750 10084 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:45609
I20260807 05:09:22.402092 10217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:45609 every 8 connection(s)
I20260807 05:09:22.402664 10084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:09:22.404781 10061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:22.392266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:56919 (local address 127.8.16.126:35103)
0807 05:09:22.392383 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:22.392387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:22.393390 (+ 1003us) server_negotiation.cc:407] Connection header received
0807 05:09:22.394514 (+ 1124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:22.394518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:22.394582 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:22.394678 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:22.395913 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.396607 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.398362 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.398680 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.398784 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:22.401486 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:22.401491 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:22.401493 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:22.401523 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:22.402999 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:22.403556 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:22.403559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:22.403560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:22.403639 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:22.403889 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:22.403891 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:22.403892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:22.404053 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:22.404124 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:22.404551 (+ 427us) server_negotiation.cc:299] Negotiation successful
0807 05:09:22.404655 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 05:09:22.405615 10108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:22.392732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:35103 (local address 127.8.16.65:56919)
0807 05:09:22.393235 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:22.393277 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:22.394219 (+ 942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:22.394848 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:22.394856 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:22.395263 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:22.395762 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:22.395777 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.396756 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.396760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:22.398202 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:22.398215 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.398827 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.398832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:22.399013 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:22.399596 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:22.399615 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:22.401302 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:22.403135 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:22.403141 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:22.403155 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:22.403470 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:22.403707 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:22.403708 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:22.403710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:22.403832 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:22.404146 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:22.404151 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:22.404364 (+ 213us) client_negotiation.cc:769] Sending connection context
0807 05:09:22.404516 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 05:09:22.404748 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":148,"threads_started":1}
I20260807 05:09:22.406839 10218 heartbeater.cc:347] Connected to a master server at 127.8.16.126:35103
I20260807 05:09:22.406972 10218 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:22.407505 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 10084
I20260807 05:09:22.407518 10218 heartbeater.cc:511] Master 127.8.16.126:35103 requested a full tablet report, sending...
I20260807 05:09:22.407619 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:09:22.407923 8257 external_mini_cluster.cc:1428] Setting key 043b394b39bc4f96b5e18d7045823ae5
I20260807 05:09:22.409076 10011 ts_manager.cc:194] Registered new tserver with Master: 7985da867e914492888a84a9ae418d56 (127.8.16.65:45609)
I20260807 05:09:22.410569 10011 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:56919
I20260807 05:09:22.415927 10061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:22.409040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:35965 (local address 127.8.16.126:35103)
0807 05:09:22.409138 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:22.409142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:22.409380 (+ 238us) server_negotiation.cc:407] Connection header received
0807 05:09:22.409557 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:22.409560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:22.409598 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:22.409642 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:22.410199 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.410893 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.411929 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.412161 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.412245 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:22.413233 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:22.413237 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:22.413240 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:22.413262 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:22.414520 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:22.414891 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:22.414894 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:22.414896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:22.414967 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:22.415206 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:22.415209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:22.415211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:22.415367 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:22.415449 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:22.415687 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 05:09:22.415783 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 05:09:22.415944 10224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:22.408998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:35103 (local address 127.8.16.65:35965)
0807 05:09:22.409306 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:22.409321 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:22.409438 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:22.409769 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:22.409771 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:22.409971 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:22.410082 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:22.410088 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.411046 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.411049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:22.411790 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:22.411801 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.412275 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.412278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:22.412365 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:22.412792 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:22.412802 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:22.413113 (+ 311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:22.414619 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:22.414623 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:22.414625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:22.414808 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:22.415044 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:22.415047 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:22.415049 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:22.415131 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:22.415454 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:22.415458 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:22.415541 (+ 83us) client_negotiation.cc:769] Sending connection context
0807 05:09:22.415625 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 05:09:22.415764 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for kudu/127.8.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
I20260807 05:09:22.433607 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:35103
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:43529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:22.539558 10227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:22.539831 10227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:22.539898 10227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:22.543951 10227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:22.544018 10227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:22.544071 10227 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 05:09:22.544096 10227 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 05:09:22.544150 10227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:09:22.549396 10227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:43529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.16.126:35103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:22.550391 10227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:22.551348 10227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:22.560319 10235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:22.560345 10232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:22.560345 10233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:22.560674 10227 server_base.cc:1034] running on GCE node
I20260807 05:09:22.561009 10227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:22.561578 10227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:22.562734 10227 hybrid_clock.cc:648] HybridClock initialized: now 1786079362562721 us; error 39 us; skew 500 ppm
Aug 07 05:09:22 dist-test-slave-49h3 krb5kdc[9966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079362, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:22.565205 10227 init.cc:377] Logged in from keytab as kudu/127.8.16.66@KRBTEST.COM (short username kudu)
I20260807 05:09:22.565982 10227 webserver.cc:460] Webserver started at http://127.8.16.66:46339/ using document root <none> and password file <none>
I20260807 05:09:22.566471 10227 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:22.566526 10227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:22.566681 10227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:22.568523 10227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "30adf7aa145c48bb977af86f3c170bef"
format_stamp: "Formatted at 2026-08-07 05:09:22 on dist-test-slave-49h3"
server_key: "dfdc40c4cf348568eb21e85c431c79aa"
server_key_iv: "c7c19437056b3412cf696f179fbb12f2"
server_key_version: "encryptionkey@0"
I20260807 05:09:22.569041 10227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "30adf7aa145c48bb977af86f3c170bef"
format_stamp: "Formatted at 2026-08-07 05:09:22 on dist-test-slave-49h3"
server_key: "dfdc40c4cf348568eb21e85c431c79aa"
server_key_iv: "c7c19437056b3412cf696f179fbb12f2"
server_key_version: "encryptionkey@0"
I20260807 05:09:22.572671 10227 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:09:22.574908 10242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:22.575984 10227 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:22.576102 10227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "30adf7aa145c48bb977af86f3c170bef"
format_stamp: "Formatted at 2026-08-07 05:09:22 on dist-test-slave-49h3"
server_key: "dfdc40c4cf348568eb21e85c431c79aa"
server_key_iv: "c7c19437056b3412cf696f179fbb12f2"
server_key_version: "encryptionkey@0"
I20260807 05:09:22.576195 10227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:22.592639 10227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:22.595530 10227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:22.595695 10227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:22.597071 10227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:22.597127 10227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:22.597165 10227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:22.597191 10227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 05:09:22 dist-test-slave-49h3 krb5kdc[9966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079362, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:22.607357 10227 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:38383
I20260807 05:09:22.607409 10361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:38383 every 8 connection(s)
I20260807 05:09:22.608405 10227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:09:22.609566 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 10227
I20260807 05:09:22.609689 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:09:22.610040 8257 external_mini_cluster.cc:1428] Setting key f5f66aeee51eaf42c10bc27669365380
I20260807 05:09:22.610714 10061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:22.597963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:60183 (local address 127.8.16.126:35103)
0807 05:09:22.598101 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:22.598104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:22.599077 (+ 973us) server_negotiation.cc:407] Connection header received
0807 05:09:22.600238 (+ 1161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:22.600241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:22.600301 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:22.600368 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:22.601793 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.602383 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.603485 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.603842 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.603969 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:22.606796 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:22.606799 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:22.606800 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:22.606826 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:22.608465 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:22.609059 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:22.609063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:22.609065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:22.609158 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:22.609533 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:22.609536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:22.609537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:22.609743 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:22.609835 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:22.610421 (+ 586us) server_negotiation.cc:299] Negotiation successful
0807 05:09:22.610547 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 05:09:22.611593 10252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:22.598444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:35103 (local address 127.8.16.66:60183)
0807 05:09:22.598924 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:22.598966 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:22.599960 (+ 994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:22.600634 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:22.600643 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:22.601031 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:22.601651 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:22.601666 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.602494 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.602497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:22.603365 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:22.603375 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.604017 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.604021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:22.604210 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:22.604760 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:22.604776 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:22.606583 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:22.608599 (+ 2016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:22.608604 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:22.608617 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:22.608944 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:22.609238 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:22.609240 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:22.609242 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:22.609419 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:22.609855 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:22.609862 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:22.610178 (+ 316us) client_negotiation.cc:769] Sending connection context
0807 05:09:22.610373 (+ 195us) client_negotiation.cc:240] Negotiation successful
0807 05:09:22.610676 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":148,"threads_started":1}
I20260807 05:09:22.612901 10362 heartbeater.cc:347] Connected to a master server at 127.8.16.126:35103
I20260807 05:09:22.613041 10362 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:22.613612 10362 heartbeater.cc:511] Master 127.8.16.126:35103 requested a full tablet report, sending...
I20260807 05:09:22.614713 10012 ts_manager.cc:194] Registered new tserver with Master: 30adf7aa145c48bb977af86f3c170bef (127.8.16.66:38383)
I20260807 05:09:22.615278 10012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:60183
WARNING: no policy specified for kudu/127.8.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.67@KRBTEST.COM" created.
I20260807 05:09:22.624752 10061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:22.617885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:59973 (local address 127.8.16.126:35103)
0807 05:09:22.617997 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:22.618001 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:22.618133 (+ 132us) server_negotiation.cc:407] Connection header received
0807 05:09:22.618256 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:22.618258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:22.618295 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:22.618347 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:22.618820 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.619314 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.620444 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.620662 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.620734 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:22.621734 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:22.621737 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:22.621739 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:22.621760 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:22.623261 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:22.623671 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:22.623673 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:22.623674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:22.623739 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:22.623968 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:22.623970 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:22.623971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:22.624183 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:22.624256 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:22.624496 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 05:09:22.624589 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 05:09:22.624794 10369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:22.617770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:35103 (local address 127.8.16.66:59973)
0807 05:09:22.618046 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:22.618060 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:22.618171 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:22.618383 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:22.618386 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:22.618599 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:22.618728 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:22.618736 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.619436 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.619440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:22.620340 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:22.620351 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.620749 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.620752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:22.620820 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:22.621272 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:22.621283 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:22.621613 (+ 330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:22.623359 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:22.623362 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:22.623364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:22.623577 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:22.623810 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:22.623812 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:22.623814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:22.623900 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:22.624266 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:22.624268 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:22.624357 (+ 89us) client_negotiation.cc:769] Sending connection context
0807 05:09:22.624461 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 05:09:22.624604 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":97,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
I20260807 05:09:22.638720 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:35103
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:43529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:22.747668 10371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:22.747993 10371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:22.748056 10371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:22.752029 10371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:22.752082 10371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:22.752121 10371 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 05:09:22.752139 10371 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 05:09:22.752168 10371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:09:22.757347 10371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:43529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.16.126:35103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:22.758332 10371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:22.759287 10371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:22.768394 10376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:22.768388 10379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:22.768393 10377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:22.768615 10371 server_base.cc:1034] running on GCE node
I20260807 05:09:22.769145 10371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:22.769801 10371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:22.770996 10371 hybrid_clock.cc:648] HybridClock initialized: now 1786079362770977 us; error 33 us; skew 500 ppm
Aug 07 05:09:22 dist-test-slave-49h3 krb5kdc[9966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079362, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:22.773725 10371 init.cc:377] Logged in from keytab as kudu/127.8.16.67@KRBTEST.COM (short username kudu)
I20260807 05:09:22.774600 10371 webserver.cc:460] Webserver started at http://127.8.16.67:40239/ using document root <none> and password file <none>
I20260807 05:09:22.775161 10371 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:22.775230 10371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:22.775413 10371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:22.777380 10371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "d9a9965cdf0f423b8663f311781b2390"
format_stamp: "Formatted at 2026-08-07 05:09:22 on dist-test-slave-49h3"
server_key: "2c52b8daa0426d617f6bc528ddf4cc7f"
server_key_iv: "a462e97a0cd11c1bb1e8a01f94d16bbc"
server_key_version: "encryptionkey@0"
I20260807 05:09:22.777935 10371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "d9a9965cdf0f423b8663f311781b2390"
format_stamp: "Formatted at 2026-08-07 05:09:22 on dist-test-slave-49h3"
server_key: "2c52b8daa0426d617f6bc528ddf4cc7f"
server_key_iv: "a462e97a0cd11c1bb1e8a01f94d16bbc"
server_key_version: "encryptionkey@0"
I20260807 05:09:22.781554 10371 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:09:22.783876 10386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:22.784997 10371 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:22.785141 10371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "d9a9965cdf0f423b8663f311781b2390"
format_stamp: "Formatted at 2026-08-07 05:09:22 on dist-test-slave-49h3"
server_key: "2c52b8daa0426d617f6bc528ddf4cc7f"
server_key_iv: "a462e97a0cd11c1bb1e8a01f94d16bbc"
server_key_version: "encryptionkey@0"
I20260807 05:09:22.785239 10371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:22.796595 10371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:22.799239 10371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:22.799417 10371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:22.800823 10371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:22.800873 10371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:22.800925 10371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:22.800953 10371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 05:09:22 dist-test-slave-49h3 krb5kdc[9966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079362, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:22.811292 10371 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:37201
I20260807 05:09:22.811651 10503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:37201 every 8 connection(s)
I20260807 05:09:22.812251 10371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:09:22.814433 10061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:22.801522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:52807 (local address 127.8.16.126:35103)
0807 05:09:22.801673 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:22.801678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:22.802871 (+ 1193us) server_negotiation.cc:407] Connection header received
0807 05:09:22.803927 (+ 1056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:22.803931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:22.803991 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:22.804074 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:22.805670 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.806293 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.807945 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.808216 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.808327 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:22.810767 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:22.810770 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:22.810773 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:22.810799 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:22.812402 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:22.813111 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:22.813114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:22.813115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:22.813196 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:22.813521 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:22.813524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:22.813525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:22.813698 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:22.813781 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:22.814210 (+ 429us) server_negotiation.cc:299] Negotiation successful
0807 05:09:22.814311 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 05:09:22.815023 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 10371
I20260807 05:09:22.815114 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:09:22.815136 10395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:22.802270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:35103 (local address 127.8.16.67:52807)
0807 05:09:22.802714 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:22.802772 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:22.803721 (+ 949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:22.804281 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:22.804290 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:22.804773 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:22.805501 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:22.805521 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.806395 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.806400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:22.807790 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:22.807802 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.808368 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.808372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:22.808558 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:22.809096 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:22.809112 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:22.810609 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:22.812505 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:22.812510 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:22.812520 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:22.812869 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:22.813279 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:22.813282 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:22.813283 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:22.813413 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:22.813780 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:22.813785 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:22.813995 (+ 210us) client_negotiation.cc:769] Sending connection context
0807 05:09:22.814163 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 05:09:22.814394 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":112,"threads_started":1}
I20260807 05:09:22.815414 8257 external_mini_cluster.cc:1428] Setting key 067892f08a68474b5541ef02f7dee655
I20260807 05:09:22.818109 10504 heartbeater.cc:347] Connected to a master server at 127.8.16.126:35103
I20260807 05:09:22.818254 10504 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:22.818708 10504 heartbeater.cc:511] Master 127.8.16.126:35103 requested a full tablet report, sending...
I20260807 05:09:22.819612 10011 ts_manager.cc:194] Registered new tserver with Master: d9a9965cdf0f423b8663f311781b2390 (127.8.16.67:37201)
I20260807 05:09:22.820255 10011 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:52807
I20260807 05:09:22.824327 10510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:22.818271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:35103 (local address 127.8.16.67:50375)
0807 05:09:22.818520 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:22.818532 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:22.818610 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:22.818955 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:22.818957 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:22.819132 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:22.819222 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:22.819227 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.820128 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.820130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:22.820758 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:22.820767 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.821266 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.821268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:22.821331 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:22.821759 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:22.821769 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:22.822042 (+ 273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:22.823299 (+ 1257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:22.823301 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:22.823303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:22.823474 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:22.823665 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:22.823667 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:22.823668 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:22.823729 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:22.823975 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:22.823976 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:22.824040 (+ 64us) client_negotiation.cc:769] Sending connection context
0807 05:09:22.824116 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 05:09:22.824209 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":91,"threads_started":1}
I20260807 05:09:22.824345 10061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:22.818644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:50375 (local address 127.8.16.126:35103)
0807 05:09:22.818771 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:22.818775 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:22.818788 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:09:22.818819 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:22.818822 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:22.818861 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:22.818937 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:22.819353 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.820016 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.820857 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.821167 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.821266 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:22.822160 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:22.822162 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:22.822164 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:22.822184 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:22.823211 (+ 1027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:22.823542 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:22.823544 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:22.823545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:22.823607 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:22.823778 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:22.823780 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:22.823781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:22.823918 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:22.823981 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:22.824163 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 05:09:22.824244 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:09:22.828626 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 05:09:22 dist-test-slave-49h3 krb5kdc[9966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 05:09:22 dist-test-slave-49h3 krb5kdc[9966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:22.853466 10061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:22.845956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48666 (local address 127.8.16.126:35103)
0807 05:09:22.846091 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:22.846094 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:22.846106 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:09:22.846144 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:22.846146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:22.846186 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:22.846252 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:22.846950 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.847464 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.848285 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.848500 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.848590 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:22.850860 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:22.850864 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:22.850866 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:22.850890 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:22.851975 (+ 1085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:22.852355 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:22.852358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:22.852359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:22.852434 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:22.852670 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:22.852673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:22.852675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:22.852853 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:22.852969 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:22.853186 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 05:09:22.853293 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:09:22.856238 10011 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48666:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:09:22.859261 10011 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:09:22.872563 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:22.867538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37026 (local address 127.8.16.66:38383)
0807 05:09:22.868015 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:22.868025 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:22.868326 (+ 301us) server_negotiation.cc:407] Connection header received
0807 05:09:22.868402 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:22.868407 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:22.868563 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:22.868707 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:22.868912 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.869670 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.871144 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.871838 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.872063 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:22.872226 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:22.872329 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 05:09:22.872414 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":154,"threads_started":1}
I20260807 05:09:22.872551 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:22.867103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:38383 (local address 127.0.0.1:37026)
0807 05:09:22.868150 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:22.868163 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:22.868244 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:22.868708 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:22.868711 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:22.868727 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:22.868814 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:22.868820 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.869770 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.869773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:22.871029 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:22.871040 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.872042 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.872046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:22.872151 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:22.872186 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 05:09:22.872300 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 05:09:22.872415 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":994,"thread_start_us":102,"threads_started":1}
I20260807 05:09:22.873000 10523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:22.866863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:45609 (local address 127.0.0.1:40954)
0807 05:09:22.867513 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:22.867564 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:22.867702 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:22.868558 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:22.868561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:22.868574 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:22.868683 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:22.868689 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.870194 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.870198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:22.871328 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:22.871335 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.872400 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.872404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:22.872500 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:22.872523 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:09:22.872584 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 05:09:22.872866 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":91,"spinlock_wait_cycles":4224,"thread_start_us":91,"threads_started":1}
I20260807 05:09:22.873677 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:22.866771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:37201 (local address 127.0.0.1:54670)
0807 05:09:22.867513 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:22.867564 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:22.867703 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:22.868493 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:22.868499 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:22.868523 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:22.868682 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:22.868689 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.870207 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.870210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:22.871398 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:22.871408 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.872584 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.872588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:22.872667 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:22.872685 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 05:09:22.872744 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 05:09:22.873548 (+ 804us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"mutex_wait_us":41,"spinlock_wait_cycles":5504,"thread_start_us":67,"threads_started":1}
I20260807 05:09:22.873737 10525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:22.867436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40954 (local address 127.8.16.65:45609)
0807 05:09:22.868112 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:22.868128 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:22.868150 (+ 22us) server_negotiation.cc:407] Connection header received
0807 05:09:22.868221 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:22.868226 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:22.868418 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:22.868827 (+ 409us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:22.868953 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.870069 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.871435 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.872266 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.873281 (+ 1015us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:22.873435 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:22.873517 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 05:09:22.873602 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":71,"threads_started":1}
I20260807 05:09:22.874235 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:22.867103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54670 (local address 127.8.16.67:37201)
0807 05:09:22.867998 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:22.868010 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:22.868034 (+ 24us) server_negotiation.cc:407] Connection header received
0807 05:09:22.868126 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:22.868132 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:22.868336 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:22.868480 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:22.868823 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.870089 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.871515 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:22.872266 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:22.872371 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:22.872517 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:22.874006 (+ 1489us) server_negotiation.cc:299] Negotiation successful
0807 05:09:22.874100 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":796,"thread_start_us":111,"threads_started":1}
I20260807 05:09:22.875638 10294 tablet_service.cc:1467] Processing CreateTablet for tablet 73ba7e71f02c44ac979d89fcb9a6c525 (DEFAULT_TABLE table=test-table [id=375df97db33f48b9b2494d8d12e21d38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:22.875917 10150 tablet_service.cc:1467] Processing CreateTablet for tablet 73ba7e71f02c44ac979d89fcb9a6c525 (DEFAULT_TABLE table=test-table [id=375df97db33f48b9b2494d8d12e21d38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:22.876772 10150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73ba7e71f02c44ac979d89fcb9a6c525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:22.876777 10294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73ba7e71f02c44ac979d89fcb9a6c525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:22.876778 10436 tablet_service.cc:1467] Processing CreateTablet for tablet 73ba7e71f02c44ac979d89fcb9a6c525 (DEFAULT_TABLE table=test-table [id=375df97db33f48b9b2494d8d12e21d38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:22.877718 10436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73ba7e71f02c44ac979d89fcb9a6c525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:22.883806 10527 tablet_bootstrap.cc:492] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56: Bootstrap starting.
I20260807 05:09:22.885229 10529 tablet_bootstrap.cc:492] T 73ba7e71f02c44ac979d89fcb9a6c525 P d9a9965cdf0f423b8663f311781b2390: Bootstrap starting.
I20260807 05:09:22.885722 10528 tablet_bootstrap.cc:492] T 73ba7e71f02c44ac979d89fcb9a6c525 P 30adf7aa145c48bb977af86f3c170bef: Bootstrap starting.
I20260807 05:09:22.886415 10527 tablet_bootstrap.cc:654] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:22.887253 10529 tablet_bootstrap.cc:654] T 73ba7e71f02c44ac979d89fcb9a6c525 P d9a9965cdf0f423b8663f311781b2390: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:22.887265 10527 log.cc:824] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:22.887320 10528 tablet_bootstrap.cc:654] T 73ba7e71f02c44ac979d89fcb9a6c525 P 30adf7aa145c48bb977af86f3c170bef: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:22.887988 10529 log.cc:824] T 73ba7e71f02c44ac979d89fcb9a6c525 P d9a9965cdf0f423b8663f311781b2390: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:22.888110 10528 log.cc:824] T 73ba7e71f02c44ac979d89fcb9a6c525 P 30adf7aa145c48bb977af86f3c170bef: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:22.889395 10527 tablet_bootstrap.cc:492] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56: No bootstrap required, opened a new log
I20260807 05:09:22.889647 10527 ts_tablet_manager.cc:1397] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 05:09:22.889938 10529 tablet_bootstrap.cc:492] T 73ba7e71f02c44ac979d89fcb9a6c525 P d9a9965cdf0f423b8663f311781b2390: No bootstrap required, opened a new log
I20260807 05:09:22.890115 10529 ts_tablet_manager.cc:1397] T 73ba7e71f02c44ac979d89fcb9a6c525 P d9a9965cdf0f423b8663f311781b2390: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:09:22.890241 10528 tablet_bootstrap.cc:492] T 73ba7e71f02c44ac979d89fcb9a6c525 P 30adf7aa145c48bb977af86f3c170bef: No bootstrap required, opened a new log
I20260807 05:09:22.890412 10528 ts_tablet_manager.cc:1397] T 73ba7e71f02c44ac979d89fcb9a6c525 P 30adf7aa145c48bb977af86f3c170bef: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 05:09:22.892562 10527 raft_consensus.cc:348] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } } peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } }
I20260807 05:09:22.892782 10527 raft_consensus.cc:374] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:22.892851 10527 raft_consensus.cc:729] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7985da867e914492888a84a9ae418d56, State: Initialized, Role: FOLLOWER
I20260807 05:09:22.893255 10527 consensus_queue.cc:260] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } } peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } }
I20260807 05:09:22.893882 10218 heartbeater.cc:503] Master 127.8.16.126:35103 was elected leader, sending a full tablet report...
I20260807 05:09:22.893863 10529 raft_consensus.cc:348] T 73ba7e71f02c44ac979d89fcb9a6c525 P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } } peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } }
I20260807 05:09:22.893909 10528 raft_consensus.cc:348] T 73ba7e71f02c44ac979d89fcb9a6c525 P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } } peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } }
I20260807 05:09:22.894101 10528 raft_consensus.cc:374] T 73ba7e71f02c44ac979d89fcb9a6c525 P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:22.894102 10529 raft_consensus.cc:374] T 73ba7e71f02c44ac979d89fcb9a6c525 P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:22.894147 10528 raft_consensus.cc:729] T 73ba7e71f02c44ac979d89fcb9a6c525 P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30adf7aa145c48bb977af86f3c170bef, State: Initialized, Role: FOLLOWER
I20260807 05:09:22.894147 10529 raft_consensus.cc:729] T 73ba7e71f02c44ac979d89fcb9a6c525 P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9a9965cdf0f423b8663f311781b2390, State: Initialized, Role: FOLLOWER
I20260807 05:09:22.894228 10527 ts_tablet_manager.cc:1428] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56: Time spent starting tablet: real 0.005s user 0.003s sys 0.001s
I20260807 05:09:22.894539 10529 consensus_queue.cc:260] T 73ba7e71f02c44ac979d89fcb9a6c525 P d9a9965cdf0f423b8663f311781b2390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } } peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } }
I20260807 05:09:22.894539 10528 consensus_queue.cc:260] T 73ba7e71f02c44ac979d89fcb9a6c525 P 30adf7aa145c48bb977af86f3c170bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } } peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } }
I20260807 05:09:22.895466 10529 ts_tablet_manager.cc:1428] T 73ba7e71f02c44ac979d89fcb9a6c525 P d9a9965cdf0f423b8663f311781b2390: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 05:09:22.895680 10528 ts_tablet_manager.cc:1428] T 73ba7e71f02c44ac979d89fcb9a6c525 P 30adf7aa145c48bb977af86f3c170bef: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 05:09:22.895843 10504 heartbeater.cc:503] Master 127.8.16.126:35103 was elected leader, sending a full tablet report...
I20260807 05:09:22.896710 10362 heartbeater.cc:503] Master 127.8.16.126:35103 was elected leader, sending a full tablet report...
W20260807 05:09:22.903818 10219 tablet.cc:2369] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 05:09:23.063274 10505 tablet.cc:2369] T 73ba7e71f02c44ac979d89fcb9a6c525 P d9a9965cdf0f423b8663f311781b2390: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 05:09:23.109582 10363 tablet.cc:2369] T 73ba7e71f02c44ac979d89fcb9a6c525 P 30adf7aa145c48bb977af86f3c170bef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:09:23.175257 10533 raft_consensus.cc:482] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:09:23.175485 10533 raft_consensus.cc:504] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } } peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } }
I20260807 05:09:23.176648 10533 leader_election.cc:288] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9a9965cdf0f423b8663f311781b2390 (127.8.16.67:37201), 30adf7aa145c48bb977af86f3c170bef (127.8.16.66:38383)
I20260807 05:09:23.180369 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.177035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:40555 (local address 127.8.16.66:38383)
0807 05:09:23.177161 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:23.177165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:23.177353 (+ 188us) server_negotiation.cc:407] Connection header received
0807 05:09:23.177499 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:23.177501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:23.177544 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:23.177608 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:23.177866 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.178373 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.179454 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.179852 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.179945 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.180006 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:23.180161 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 05:09:23.180226 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 05:09:23.180409 10537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.176938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:38383 (local address 127.8.16.65:40555)
0807 05:09:23.177269 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:23.177280 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:23.177380 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:23.177632 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:23.177635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:23.177650 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:23.177767 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.177772 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.178478 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.178481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.179346 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.179356 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.179960 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.179963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.180056 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.180075 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 05:09:23.180138 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 05:09:23.180207 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":21,"thread_start_us":81,"threads_started":1}
I20260807 05:09:23.180410 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.176996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:43163 (local address 127.8.16.67:37201)
0807 05:09:23.177200 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:23.177205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:23.177277 (+ 72us) server_negotiation.cc:407] Connection header received
0807 05:09:23.177419 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:23.177422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:23.177501 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:23.177565 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:23.177821 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.178323 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.179441 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.179882 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.179973 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.180034 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:23.180214 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 05:09:23.180295 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260807 05:09:23.180435 10536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.176873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:37201 (local address 127.8.16.65:43163)
0807 05:09:23.177189 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:23.177203 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:23.177320 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:23.177586 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:23.177588 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:23.177604 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:23.177718 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.177723 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.178431 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.178434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.179326 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.179335 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.179989 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.179993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.180108 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.180126 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 05:09:23.180188 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 05:09:23.180254 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":34,"thread_start_us":103,"threads_started":1}
I20260807 05:09:23.181041 10314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ba7e71f02c44ac979d89fcb9a6c525" candidate_uuid: "7985da867e914492888a84a9ae418d56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30adf7aa145c48bb977af86f3c170bef" is_pre_election: true
I20260807 05:09:23.181056 10457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ba7e71f02c44ac979d89fcb9a6c525" candidate_uuid: "7985da867e914492888a84a9ae418d56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9a9965cdf0f423b8663f311781b2390" is_pre_election: true
I20260807 05:09:23.181324 10457 raft_consensus.cc:2454] T 73ba7e71f02c44ac979d89fcb9a6c525 P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7985da867e914492888a84a9ae418d56 in term 0.
I20260807 05:09:23.181329 10314 raft_consensus.cc:2454] T 73ba7e71f02c44ac979d89fcb9a6c525 P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7985da867e914492888a84a9ae418d56 in term 0.
I20260807 05:09:23.181787 10102 leader_election.cc:302] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30adf7aa145c48bb977af86f3c170bef, 7985da867e914492888a84a9ae418d56; no voters:
I20260807 05:09:23.182035 10533 raft_consensus.cc:2781] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:09:23.182122 10533 raft_consensus.cc:482] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:09:23.182165 10533 raft_consensus.cc:3037] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:23.183178 10533 raft_consensus.cc:504] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } } peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } }
I20260807 05:09:23.183467 10533 leader_election.cc:288] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56 [CANDIDATE]: Term 1 election: Requested vote from peers d9a9965cdf0f423b8663f311781b2390 (127.8.16.67:37201), 30adf7aa145c48bb977af86f3c170bef (127.8.16.66:38383)
I20260807 05:09:23.183769 10457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ba7e71f02c44ac979d89fcb9a6c525" candidate_uuid: "7985da867e914492888a84a9ae418d56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9a9965cdf0f423b8663f311781b2390"
I20260807 05:09:23.183789 10314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ba7e71f02c44ac979d89fcb9a6c525" candidate_uuid: "7985da867e914492888a84a9ae418d56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30adf7aa145c48bb977af86f3c170bef"
I20260807 05:09:23.183884 10314 raft_consensus.cc:3037] T 73ba7e71f02c44ac979d89fcb9a6c525 P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:23.183878 10457 raft_consensus.cc:3037] T 73ba7e71f02c44ac979d89fcb9a6c525 P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:23.184844 10314 raft_consensus.cc:2454] T 73ba7e71f02c44ac979d89fcb9a6c525 P 30adf7aa145c48bb977af86f3c170bef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7985da867e914492888a84a9ae418d56 in term 1.
I20260807 05:09:23.185103 10102 leader_election.cc:302] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30adf7aa145c48bb977af86f3c170bef, 7985da867e914492888a84a9ae418d56; no voters:
I20260807 05:09:23.185258 10533 raft_consensus.cc:2781] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:23.185473 10533 raft_consensus.cc:686] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56 [term 1 LEADER]: Becoming Leader. State: Replica: 7985da867e914492888a84a9ae418d56, State: Running, Role: LEADER
I20260807 05:09:23.185544 10457 raft_consensus.cc:2454] T 73ba7e71f02c44ac979d89fcb9a6c525 P d9a9965cdf0f423b8663f311781b2390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7985da867e914492888a84a9ae418d56 in term 1.
I20260807 05:09:23.185817 10533 consensus_queue.cc:237] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } } peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } }
I20260807 05:09:23.188436 10010 catalog_manager.cc:5495] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7985da867e914492888a84a9ae418d56 (127.8.16.65). New cstate: current_term: 1 leader_uuid: "7985da867e914492888a84a9ae418d56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } health_report { overall_health: UNKNOWN } } }
Aug 07 05:09:23 dist-test-slave-49h3 krb5kdc[9966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079362, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:23.259203 10061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.251304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48670 (local address 127.8.16.126:35103)
0807 05:09:23.251433 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:23.251438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:23.251585 (+ 147us) server_negotiation.cc:407] Connection header received
0807 05:09:23.251709 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:23.251712 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:23.251750 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:23.251813 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:23.252493 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.252997 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.253798 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.254000 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.254082 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.256381 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:23.256385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:23.256387 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:23.256408 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:23.257619 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:23.258035 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:23.258038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:23.258040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:23.258115 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:23.258345 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:23.258348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:23.258350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:23.258518 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:23.258610 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:23.258879 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 05:09:23.259048 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":48}
I20260807 05:09:23.259203 10545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.251228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:35103 (local address 127.0.0.1:48670)
0807 05:09:23.251502 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:23.251514 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:23.251631 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:23.251845 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:23.251848 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:23.252283 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:23.252385 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.252391 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.253097 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.253100 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.253698 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.253708 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.254098 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.254102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.254180 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.254727 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:23.254740 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:23.256239 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:23.257711 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:23.257721 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:23.257728 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:23.257946 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:23.258193 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:23.258196 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:23.258198 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:23.258270 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:23.258611 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:23.258619 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:23.258727 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 05:09:23.258822 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 05:09:23.258997 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":92,"threads_started":1}
I20260807 05:09:23.263114 10010 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:48670:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 05:09:23.263772 10010 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:09:23.268294 10294 tablet_service.cc:1467] Processing CreateTablet for tablet d5a9d88c72dd478fafa6aa258dd236ad (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5869b7f57dcf47b2b529be6838f63f28]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 05:09:23.268226 10150 tablet_service.cc:1467] Processing CreateTablet for tablet d5a9d88c72dd478fafa6aa258dd236ad (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5869b7f57dcf47b2b529be6838f63f28]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 05:09:23.268395 10436 tablet_service.cc:1467] Processing CreateTablet for tablet d5a9d88c72dd478fafa6aa258dd236ad (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5869b7f57dcf47b2b529be6838f63f28]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 05:09:23.268970 10150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5a9d88c72dd478fafa6aa258dd236ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:23.268970 10294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5a9d88c72dd478fafa6aa258dd236ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:23.269066 10436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5a9d88c72dd478fafa6aa258dd236ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:23.271657 10529 tablet_bootstrap.cc:492] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390: Bootstrap starting.
I20260807 05:09:23.271788 10528 tablet_bootstrap.cc:492] T d5a9d88c72dd478fafa6aa258dd236ad P 30adf7aa145c48bb977af86f3c170bef: Bootstrap starting.
I20260807 05:09:23.272487 10529 tablet_bootstrap.cc:654] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:23.272570 10528 tablet_bootstrap.cc:654] T d5a9d88c72dd478fafa6aa258dd236ad P 30adf7aa145c48bb977af86f3c170bef: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:23.273267 10527 tablet_bootstrap.cc:492] T d5a9d88c72dd478fafa6aa258dd236ad P 7985da867e914492888a84a9ae418d56: Bootstrap starting.
I20260807 05:09:23.273720 10529 tablet_bootstrap.cc:492] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390: No bootstrap required, opened a new log
I20260807 05:09:23.273801 10529 ts_tablet_manager.cc:1397] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:23.274220 10529 raft_consensus.cc:348] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:23.274315 10529 raft_consensus.cc:374] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:23.274335 10529 raft_consensus.cc:729] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9a9965cdf0f423b8663f311781b2390, State: Initialized, Role: FOLLOWER
I20260807 05:09:23.274451 10529 consensus_queue.cc:260] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:23.274571 10527 tablet_bootstrap.cc:654] T d5a9d88c72dd478fafa6aa258dd236ad P 7985da867e914492888a84a9ae418d56: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:23.274701 10529 ts_tablet_manager.cc:1428] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 05:09:23.274717 10534 tablet_replica.cc:439] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } } }
I20260807 05:09:23.274865 10534 tablet_replica.cc:442] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:09:23.275297 10528 tablet_bootstrap.cc:492] T d5a9d88c72dd478fafa6aa258dd236ad P 30adf7aa145c48bb977af86f3c170bef: No bootstrap required, opened a new log
I20260807 05:09:23.275375 10528 ts_tablet_manager.cc:1397] T d5a9d88c72dd478fafa6aa258dd236ad P 30adf7aa145c48bb977af86f3c170bef: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 05:09:23.275751 10528 raft_consensus.cc:348] T d5a9d88c72dd478fafa6aa258dd236ad P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:23.275853 10528 raft_consensus.cc:374] T d5a9d88c72dd478fafa6aa258dd236ad P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:23.275887 10528 raft_consensus.cc:729] T d5a9d88c72dd478fafa6aa258dd236ad P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30adf7aa145c48bb977af86f3c170bef, State: Initialized, Role: FOLLOWER
I20260807 05:09:23.276005 10528 consensus_queue.cc:260] T d5a9d88c72dd478fafa6aa258dd236ad P 30adf7aa145c48bb977af86f3c170bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:23.276252 10528 ts_tablet_manager.cc:1428] T d5a9d88c72dd478fafa6aa258dd236ad P 30adf7aa145c48bb977af86f3c170bef: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 05:09:23.276271 10535 tablet_replica.cc:439] T d5a9d88c72dd478fafa6aa258dd236ad P 30adf7aa145c48bb977af86f3c170bef: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } } }
I20260807 05:09:23.276412 10535 tablet_replica.cc:442] T d5a9d88c72dd478fafa6aa258dd236ad P 30adf7aa145c48bb977af86f3c170bef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:09:23.277541 10527 tablet_bootstrap.cc:492] T d5a9d88c72dd478fafa6aa258dd236ad P 7985da867e914492888a84a9ae418d56: No bootstrap required, opened a new log
I20260807 05:09:23.277633 10527 ts_tablet_manager.cc:1397] T d5a9d88c72dd478fafa6aa258dd236ad P 7985da867e914492888a84a9ae418d56: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260807 05:09:23.278014 10527 raft_consensus.cc:348] T d5a9d88c72dd478fafa6aa258dd236ad P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:23.278138 10527 raft_consensus.cc:374] T d5a9d88c72dd478fafa6aa258dd236ad P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:23.278180 10527 raft_consensus.cc:729] T d5a9d88c72dd478fafa6aa258dd236ad P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7985da867e914492888a84a9ae418d56, State: Initialized, Role: FOLLOWER
I20260807 05:09:23.278296 10527 consensus_queue.cc:260] T d5a9d88c72dd478fafa6aa258dd236ad P 7985da867e914492888a84a9ae418d56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:23.278523 10527 ts_tablet_manager.cc:1428] T d5a9d88c72dd478fafa6aa258dd236ad P 7985da867e914492888a84a9ae418d56: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 05:09:23.278558 10533 tablet_replica.cc:439] T d5a9d88c72dd478fafa6aa258dd236ad P 7985da867e914492888a84a9ae418d56: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } } }
I20260807 05:09:23.278697 10533 tablet_replica.cc:442] T d5a9d88c72dd478fafa6aa258dd236ad P 7985da867e914492888a84a9ae418d56: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:09:23.573426 10534 raft_consensus.cc:482] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:09:23.573599 10534 raft_consensus.cc:504] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:23.574693 10534 leader_election.cc:288] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7985da867e914492888a84a9ae418d56 (127.8.16.65:45609), 30adf7aa145c48bb977af86f3c170bef (127.8.16.66:38383)
I20260807 05:09:23.578421 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.575039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:42163 (local address 127.8.16.66:38383)
0807 05:09:23.575213 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:23.575218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:23.575364 (+ 146us) server_negotiation.cc:407] Connection header received
0807 05:09:23.575477 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:23.575480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:23.575536 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:23.575608 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:23.575863 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.576377 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.577508 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.577894 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.577983 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.578041 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:23.578237 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 05:09:23.578311 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260807 05:09:23.578426 10551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.574989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:50091 (local address 127.8.16.65:45609)
0807 05:09:23.575213 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:23.575230 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:23.575322 (+ 92us) server_negotiation.cc:407] Connection header received
0807 05:09:23.575457 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:23.575460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:23.575509 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:23.575586 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:23.575872 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.576374 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.577499 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.577889 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.577978 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.578042 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:23.578230 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 05:09:23.578295 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":56,"threads_started":1}
I20260807 05:09:23.578431 10549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.574916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:45609 (local address 127.8.16.67:50091)
0807 05:09:23.575210 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:23.575224 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:23.575350 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:23.575606 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:23.575608 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:23.575624 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:23.575763 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.575768 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.576482 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.576485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.577358 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.577367 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.577996 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.577999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.578120 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.578139 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 05:09:23.578200 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 05:09:23.578271 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":11,"spinlock_wait_cycles":1408,"thread_start_us":92,"threads_started":1}
I20260807 05:09:23.578440 10550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.574945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:38383 (local address 127.8.16.67:42163)
0807 05:09:23.575274 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:23.575287 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:23.575391 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:23.575632 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:23.575634 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:23.575668 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:23.575764 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.575769 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.576484 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.576487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.577357 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.577366 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.578006 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.578010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.578128 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.578146 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 05:09:23.578211 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 05:09:23.578280 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":98,"thread_start_us":69,"threads_started":1}
I20260807 05:09:23.578980 10171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5a9d88c72dd478fafa6aa258dd236ad" candidate_uuid: "d9a9965cdf0f423b8663f311781b2390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7985da867e914492888a84a9ae418d56" is_pre_election: true
I20260807 05:09:23.579102 10314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5a9d88c72dd478fafa6aa258dd236ad" candidate_uuid: "d9a9965cdf0f423b8663f311781b2390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30adf7aa145c48bb977af86f3c170bef" is_pre_election: true
I20260807 05:09:23.579221 10171 raft_consensus.cc:2454] T d5a9d88c72dd478fafa6aa258dd236ad P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9a9965cdf0f423b8663f311781b2390 in term 0.
I20260807 05:09:23.579263 10314 raft_consensus.cc:2454] T d5a9d88c72dd478fafa6aa258dd236ad P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9a9965cdf0f423b8663f311781b2390 in term 0.
I20260807 05:09:23.579591 10389 leader_election.cc:302] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30adf7aa145c48bb977af86f3c170bef, d9a9965cdf0f423b8663f311781b2390; no voters:
I20260807 05:09:23.579846 10534 raft_consensus.cc:2781] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:09:23.579910 10534 raft_consensus.cc:482] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:09:23.579939 10534 raft_consensus.cc:3037] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:23.580950 10534 raft_consensus.cc:504] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:23.581282 10534 leader_election.cc:288] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390 [CANDIDATE]: Term 1 election: Requested vote from peers 7985da867e914492888a84a9ae418d56 (127.8.16.65:45609), 30adf7aa145c48bb977af86f3c170bef (127.8.16.66:38383)
I20260807 05:09:23.581599 10171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5a9d88c72dd478fafa6aa258dd236ad" candidate_uuid: "d9a9965cdf0f423b8663f311781b2390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7985da867e914492888a84a9ae418d56"
I20260807 05:09:23.581624 10314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5a9d88c72dd478fafa6aa258dd236ad" candidate_uuid: "d9a9965cdf0f423b8663f311781b2390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30adf7aa145c48bb977af86f3c170bef"
I20260807 05:09:23.581707 10171 raft_consensus.cc:3037] T d5a9d88c72dd478fafa6aa258dd236ad P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:23.581730 10314 raft_consensus.cc:3037] T d5a9d88c72dd478fafa6aa258dd236ad P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:23.582629 10171 raft_consensus.cc:2454] T d5a9d88c72dd478fafa6aa258dd236ad P 7985da867e914492888a84a9ae418d56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9a9965cdf0f423b8663f311781b2390 in term 1.
I20260807 05:09:23.582656 10314 raft_consensus.cc:2454] T d5a9d88c72dd478fafa6aa258dd236ad P 30adf7aa145c48bb977af86f3c170bef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9a9965cdf0f423b8663f311781b2390 in term 1.
I20260807 05:09:23.582880 10390 leader_election.cc:302] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7985da867e914492888a84a9ae418d56, d9a9965cdf0f423b8663f311781b2390; no voters:
I20260807 05:09:23.583040 10534 raft_consensus.cc:2781] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:23.583217 10534 raft_consensus.cc:686] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390 [term 1 LEADER]: Becoming Leader. State: Replica: d9a9965cdf0f423b8663f311781b2390, State: Running, Role: LEADER
I20260807 05:09:23.583546 10534 consensus_queue.cc:237] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:23.585840 10552 tablet_replica.cc:439] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390: TxnStatusTablet state changed. Reason: New leader d9a9965cdf0f423b8663f311781b2390. Latest consensus state: current_term: 1 leader_uuid: "d9a9965cdf0f423b8663f311781b2390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } } }
I20260807 05:09:23.585954 10552 tablet_replica.cc:442] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390: This TxnStatusTablet replica's current role is: LEADER
I20260807 05:09:23.586236 10554 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 05:09:23.586417 10554 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 05:09:23.586640 10012 catalog_manager.cc:5495] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390 reported cstate change: term changed from 0 to 1, leader changed from <none> to d9a9965cdf0f423b8663f311781b2390 (127.8.16.67). New cstate: current_term: 1 leader_uuid: "d9a9965cdf0f423b8663f311781b2390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } health_report { overall_health: HEALTHY } } }
I20260807 05:09:23.612528 10533 consensus_queue.cc:1035] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:23.618223 10533 consensus_queue.cc:1035] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:23.635625 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.632204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54686 (local address 127.8.16.67:37201)
0807 05:09:23.632334 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:23.632339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:23.632351 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:09:23.632522 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:23.632525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:23.632575 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:23.632657 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:23.632884 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.633400 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.634249 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.634462 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.634547 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.634840 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:23.635254 (+ 414us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:23.635353 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:23.635432 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 05:09:23.635485 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:09:23.635635 10545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.632120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:37201 (local address 127.0.0.1:54686)
0807 05:09:23.632270 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:23.632286 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:23.632390 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:23.632674 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:23.632677 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:23.632684 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 05:09:23.632786 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.632792 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.633519 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.633522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.634136 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.634144 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.634554 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.634557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.634652 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.634695 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 05:09:23.635344 (+ 649us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 05:09:23.635359 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 05:09:23.635422 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 05:09:23.635507 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 05:09:23.639905 10314 raft_consensus.cc:1261] T d5a9d88c72dd478fafa6aa258dd236ad P 30adf7aa145c48bb977af86f3c170bef [term 1 FOLLOWER]: Refusing update from remote peer d9a9965cdf0f423b8663f311781b2390: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:23.640017 10171 raft_consensus.cc:1261] T d5a9d88c72dd478fafa6aa258dd236ad P 7985da867e914492888a84a9ae418d56 [term 1 FOLLOWER]: Refusing update from remote peer d9a9965cdf0f423b8663f311781b2390: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:23.640421 10552 consensus_queue.cc:1035] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:23.640658 10534 consensus_queue.cc:1035] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:23.643213 10535 tablet_replica.cc:439] T d5a9d88c72dd478fafa6aa258dd236ad P 30adf7aa145c48bb977af86f3c170bef: TxnStatusTablet state changed. Reason: New leader d9a9965cdf0f423b8663f311781b2390. Latest consensus state: current_term: 1 leader_uuid: "d9a9965cdf0f423b8663f311781b2390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } } }
I20260807 05:09:23.643352 10535 tablet_replica.cc:442] T d5a9d88c72dd478fafa6aa258dd236ad P 30adf7aa145c48bb977af86f3c170bef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:09:23.645175 10558 tablet_replica.cc:439] T d5a9d88c72dd478fafa6aa258dd236ad P 7985da867e914492888a84a9ae418d56: TxnStatusTablet state changed. Reason: New leader d9a9965cdf0f423b8663f311781b2390. Latest consensus state: current_term: 1 leader_uuid: "d9a9965cdf0f423b8663f311781b2390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } } }
I20260807 05:09:23.645291 10558 tablet_replica.cc:442] T d5a9d88c72dd478fafa6aa258dd236ad P 7985da867e914492888a84a9ae418d56: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:09:23.646791 10534 tablet_replica.cc:439] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d9a9965cdf0f423b8663f311781b2390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } } }
I20260807 05:09:23.646900 10534 tablet_replica.cc:442] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390: This TxnStatusTablet replica's current role is: LEADER
I20260807 05:09:23.646993 10534 tablet_replica.cc:439] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d9a9965cdf0f423b8663f311781b2390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } } }
I20260807 05:09:23.647056 10534 tablet_replica.cc:442] T d5a9d88c72dd478fafa6aa258dd236ad P d9a9965cdf0f423b8663f311781b2390: This TxnStatusTablet replica's current role is: LEADER
I20260807 05:09:23.647173 10535 tablet_replica.cc:439] T d5a9d88c72dd478fafa6aa258dd236ad P 30adf7aa145c48bb977af86f3c170bef: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d9a9965cdf0f423b8663f311781b2390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } } }
I20260807 05:09:23.647267 10535 tablet_replica.cc:442] T d5a9d88c72dd478fafa6aa258dd236ad P 30adf7aa145c48bb977af86f3c170bef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:09:23.647868 10559 mvcc.cc:205] Tried to move back new op lower bound from 7315781073463631872 to 7315781073240563712. Current Snapshot: MvccSnapshot[applied={T|T < 7315781073463631872}]
I20260807 05:09:23.650475 10561 mvcc.cc:205] Tried to move back new op lower bound from 7315781073463631872 to 7315781073240563712. Current Snapshot: MvccSnapshot[applied={T|T < 7315781073463631872 or (T in {7315781073463631872})}]
I20260807 05:09:23.650940 10558 tablet_replica.cc:439] T d5a9d88c72dd478fafa6aa258dd236ad P 7985da867e914492888a84a9ae418d56: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d9a9965cdf0f423b8663f311781b2390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } } }
I20260807 05:09:23.651052 10558 tablet_replica.cc:442] T d5a9d88c72dd478fafa6aa258dd236ad P 7985da867e914492888a84a9ae418d56: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:09:23.656955 10551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.652852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40964 (local address 127.8.16.65:45609)
0807 05:09:23.652976 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:23.652980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:23.653168 (+ 188us) server_negotiation.cc:407] Connection header received
0807 05:09:23.653315 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:23.653317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:23.653364 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:23.653457 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:23.653761 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.654282 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.655527 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.655764 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.655848 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.656155 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:23.656566 (+ 411us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:23.656674 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:23.656779 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 05:09:23.656835 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 05:09:23.666966 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.663666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:51175 (local address 127.8.16.67:37201)
0807 05:09:23.663784 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:23.663788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:23.663907 (+ 119us) server_negotiation.cc:407] Connection header received
0807 05:09:23.664025 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:23.664027 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:23.664068 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:23.664126 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:23.664375 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.664843 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.666043 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.666256 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.666324 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.666627 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:23.666687 (+ 60us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:23.666728 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:23.666830 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 05:09:23.666868 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 05:09:23.667054 10572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.663523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:37201 (local address 127.8.16.65:51175)
0807 05:09:23.663826 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:23.663841 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:23.663941 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:23.664149 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:23.664152 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:23.664159 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 05:09:23.664281 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.664287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.664926 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.664929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.665945 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.665957 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.666342 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.666346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.666437 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.666487 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 05:09:23.666757 (+ 270us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 05:09:23.666763 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 05:09:23.666824 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 05:09:23.666899 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":111,"threads_started":1}
I20260807 05:09:23.691475 10575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.687708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:45609 (local address 127.8.16.67:39479)
0807 05:09:23.688067 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:23.688079 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:23.688182 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:23.688403 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:23.688405 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:23.688410 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 05:09:23.688498 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.688502 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.689532 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.689536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.690377 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.690385 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.690791 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.690794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.690874 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.690920 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 05:09:23.691234 (+ 314us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 05:09:23.691240 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 05:09:23.691288 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 05:09:23.691348 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":105,"threads_started":1}
I20260807 05:09:23.691479 10551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.687775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:39479 (local address 127.8.16.65:45609)
0807 05:09:23.687887 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:23.687892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:23.688172 (+ 280us) server_negotiation.cc:407] Connection header received
0807 05:09:23.688265 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:23.688268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:23.688318 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:23.688401 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:23.688589 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.689402 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.690479 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.690696 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.690782 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.691058 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:23.691145 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:23.691219 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:23.691301 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 05:09:23.691354 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 05:09:23.714955 10012 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:23.715162 10012 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:23.717988 10012 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e: Sending DeleteTablet for 3 replicas of tablet 73ba7e71f02c44ac979d89fcb9a6c525
I20260807 05:09:23.718737 10436 tablet_service.cc:1514] Processing DeleteTablet for tablet 73ba7e71f02c44ac979d89fcb9a6c525 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:23 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:54670
I20260807 05:09:23.719048 10150 tablet_service.cc:1514] Processing DeleteTablet for tablet 73ba7e71f02c44ac979d89fcb9a6c525 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:23 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:40954
I20260807 05:09:23.719435 10578 tablet_replica.cc:330] T 73ba7e71f02c44ac979d89fcb9a6c525 P d9a9965cdf0f423b8663f311781b2390: stopping tablet replica
I20260807 05:09:23.719617 10294 tablet_service.cc:1514] Processing DeleteTablet for tablet 73ba7e71f02c44ac979d89fcb9a6c525 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:23 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:37026
I20260807 05:09:23.719883 10578 raft_consensus.cc:2229] T 73ba7e71f02c44ac979d89fcb9a6c525 P d9a9965cdf0f423b8663f311781b2390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:23.720093 10578 raft_consensus.cc:2258] T 73ba7e71f02c44ac979d89fcb9a6c525 P d9a9965cdf0f423b8663f311781b2390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:23.720916 10012 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48666:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:09:23.721341 10012 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:09:23.721872 10578 ts_tablet_manager.cc:1905] T 73ba7e71f02c44ac979d89fcb9a6c525 P d9a9965cdf0f423b8663f311781b2390: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:23.724992 10578 ts_tablet_manager.cc:1918] T 73ba7e71f02c44ac979d89fcb9a6c525 P d9a9965cdf0f423b8663f311781b2390: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:09:23.725157 10578 log.cc:1196] T 73ba7e71f02c44ac979d89fcb9a6c525 P d9a9965cdf0f423b8663f311781b2390: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/73ba7e71f02c44ac979d89fcb9a6c525
I20260807 05:09:23.725529 10578 ts_tablet_manager.cc:1939] T 73ba7e71f02c44ac979d89fcb9a6c525 P d9a9965cdf0f423b8663f311781b2390: Deleting consensus metadata
I20260807 05:09:23.726581 10150 tablet_service.cc:1467] Processing CreateTablet for tablet b6ce8123e4d0489eb5cb989db4279c63 (DEFAULT_TABLE table=test-table [id=ebb37b8fb91f4bfc9ee400d648b3849d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:23.726883 10150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6ce8123e4d0489eb5cb989db4279c63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:23.729338 10527 tablet_bootstrap.cc:492] T b6ce8123e4d0489eb5cb989db4279c63 P 7985da867e914492888a84a9ae418d56: Bootstrap starting.
I20260807 05:09:23.729915 10294 tablet_service.cc:1467] Processing CreateTablet for tablet b6ce8123e4d0489eb5cb989db4279c63 (DEFAULT_TABLE table=test-table [id=ebb37b8fb91f4bfc9ee400d648b3849d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:23.730160 10294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6ce8123e4d0489eb5cb989db4279c63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:23.730324 10527 tablet_bootstrap.cc:654] T b6ce8123e4d0489eb5cb989db4279c63 P 7985da867e914492888a84a9ae418d56: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:23.731645 10527 tablet_bootstrap.cc:492] T b6ce8123e4d0489eb5cb989db4279c63 P 7985da867e914492888a84a9ae418d56: No bootstrap required, opened a new log
I20260807 05:09:23.731724 10527 ts_tablet_manager.cc:1397] T b6ce8123e4d0489eb5cb989db4279c63 P 7985da867e914492888a84a9ae418d56: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:23.732120 10527 raft_consensus.cc:348] T b6ce8123e4d0489eb5cb989db4279c63 P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:23.732219 10527 raft_consensus.cc:374] T b6ce8123e4d0489eb5cb989db4279c63 P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:23.732246 10527 raft_consensus.cc:729] T b6ce8123e4d0489eb5cb989db4279c63 P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7985da867e914492888a84a9ae418d56, State: Initialized, Role: FOLLOWER
I20260807 05:09:23.732378 10528 tablet_bootstrap.cc:492] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef: Bootstrap starting.
I20260807 05:09:23.732355 10527 consensus_queue.cc:260] T b6ce8123e4d0489eb5cb989db4279c63 P 7985da867e914492888a84a9ae418d56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:23.732601 10527 ts_tablet_manager.cc:1428] T b6ce8123e4d0489eb5cb989db4279c63 P 7985da867e914492888a84a9ae418d56: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:09:23.733239 10528 tablet_bootstrap.cc:654] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:23.734174 9996 catalog_manager.cc:4841] TS d9a9965cdf0f423b8663f311781b2390 (127.8.16.67:37201): tablet 73ba7e71f02c44ac979d89fcb9a6c525 (table test-table [id=375df97db33f48b9b2494d8d12e21d38]) successfully deleted
I20260807 05:09:23.734704 10436 tablet_service.cc:1467] Processing CreateTablet for tablet b6ce8123e4d0489eb5cb989db4279c63 (DEFAULT_TABLE table=test-table [id=ebb37b8fb91f4bfc9ee400d648b3849d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:23.734946 10436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6ce8123e4d0489eb5cb989db4279c63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:23.736172 10528 tablet_bootstrap.cc:492] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef: No bootstrap required, opened a new log
I20260807 05:09:23.736248 10528 ts_tablet_manager.cc:1397] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 05:09:23.736642 10528 raft_consensus.cc:348] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:23.736749 10528 raft_consensus.cc:374] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:23.736778 10528 raft_consensus.cc:729] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30adf7aa145c48bb977af86f3c170bef, State: Initialized, Role: FOLLOWER
I20260807 05:09:23.736903 10528 consensus_queue.cc:260] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:23.737128 10529 tablet_bootstrap.cc:492] T b6ce8123e4d0489eb5cb989db4279c63 P d9a9965cdf0f423b8663f311781b2390: Bootstrap starting.
I20260807 05:09:23.737138 10528 ts_tablet_manager.cc:1428] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:09:23.738059 10529 tablet_bootstrap.cc:654] T b6ce8123e4d0489eb5cb989db4279c63 P d9a9965cdf0f423b8663f311781b2390: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:23.738274 10580 tablet_replica.cc:330] T 73ba7e71f02c44ac979d89fcb9a6c525 P 30adf7aa145c48bb977af86f3c170bef: stopping tablet replica
I20260807 05:09:23.738601 10580 raft_consensus.cc:2229] T 73ba7e71f02c44ac979d89fcb9a6c525 P 30adf7aa145c48bb977af86f3c170bef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:23.738817 10580 raft_consensus.cc:2258] T 73ba7e71f02c44ac979d89fcb9a6c525 P 30adf7aa145c48bb977af86f3c170bef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:23.739256 10579 tablet_replica.cc:330] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56: stopping tablet replica
I20260807 05:09:23.740362 10529 tablet_bootstrap.cc:492] T b6ce8123e4d0489eb5cb989db4279c63 P d9a9965cdf0f423b8663f311781b2390: No bootstrap required, opened a new log
I20260807 05:09:23.740439 10529 ts_tablet_manager.cc:1397] T b6ce8123e4d0489eb5cb989db4279c63 P d9a9965cdf0f423b8663f311781b2390: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 05:09:23.740814 10529 raft_consensus.cc:348] T b6ce8123e4d0489eb5cb989db4279c63 P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:23.740913 10529 raft_consensus.cc:374] T b6ce8123e4d0489eb5cb989db4279c63 P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:23.740940 10529 raft_consensus.cc:729] T b6ce8123e4d0489eb5cb989db4279c63 P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9a9965cdf0f423b8663f311781b2390, State: Initialized, Role: FOLLOWER
I20260807 05:09:23.741045 10529 consensus_queue.cc:260] T b6ce8123e4d0489eb5cb989db4279c63 P d9a9965cdf0f423b8663f311781b2390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:23.741278 10529 ts_tablet_manager.cc:1428] T b6ce8123e4d0489eb5cb989db4279c63 P d9a9965cdf0f423b8663f311781b2390: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 05:09:23.741279 10579 raft_consensus.cc:2229] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:09:23.741678 10579 raft_consensus.cc:2258] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:23.744221 10579 ts_tablet_manager.cc:1905] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:23.747500 10579 ts_tablet_manager.cc:1918] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:09:23.747609 10579 log.cc:1196] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-0/wal/wals/73ba7e71f02c44ac979d89fcb9a6c525
I20260807 05:09:23.747961 10579 ts_tablet_manager.cc:1939] T 73ba7e71f02c44ac979d89fcb9a6c525 P 7985da867e914492888a84a9ae418d56: Deleting consensus metadata
I20260807 05:09:23.749136 9998 catalog_manager.cc:4841] TS 7985da867e914492888a84a9ae418d56 (127.8.16.65:45609): tablet 73ba7e71f02c44ac979d89fcb9a6c525 (table test-table [id=375df97db33f48b9b2494d8d12e21d38]) successfully deleted
I20260807 05:09:23.749258 10580 ts_tablet_manager.cc:1905] T 73ba7e71f02c44ac979d89fcb9a6c525 P 30adf7aa145c48bb977af86f3c170bef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:23.752169 10580 ts_tablet_manager.cc:1918] T 73ba7e71f02c44ac979d89fcb9a6c525 P 30adf7aa145c48bb977af86f3c170bef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:09:23.752292 10580 log.cc:1196] T 73ba7e71f02c44ac979d89fcb9a6c525 P 30adf7aa145c48bb977af86f3c170bef: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-1/wal/wals/73ba7e71f02c44ac979d89fcb9a6c525
I20260807 05:09:23.752585 10580 ts_tablet_manager.cc:1939] T 73ba7e71f02c44ac979d89fcb9a6c525 P 30adf7aa145c48bb977af86f3c170bef: Deleting consensus metadata
I20260807 05:09:23.753791 9997 catalog_manager.cc:4841] TS 30adf7aa145c48bb977af86f3c170bef (127.8.16.66:38383): tablet 73ba7e71f02c44ac979d89fcb9a6c525 (table test-table [id=375df97db33f48b9b2494d8d12e21d38]) successfully deleted
I20260807 05:09:23.774751 10535 raft_consensus.cc:482] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:09:23.774914 10535 raft_consensus.cc:504] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:23.776208 10535 leader_election.cc:288] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7985da867e914492888a84a9ae418d56 (127.8.16.65:45609), d9a9965cdf0f423b8663f311781b2390 (127.8.16.67:37201)
I20260807 05:09:23.780774 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.776814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:57307 (local address 127.8.16.67:37201)
0807 05:09:23.776935 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:23.776939 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:23.776953 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:09:23.777004 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:23.777007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:23.777050 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:23.777137 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:23.777438 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.778243 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.779698 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.780152 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.780248 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.780316 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:23.780559 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 05:09:23.780651 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 05:09:23.780793 10585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.776520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:37201 (local address 127.8.16.66:57307)
0807 05:09:23.776820 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:23.776833 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:23.776915 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:23.777135 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:23.777139 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:23.777155 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:23.777290 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.777298 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.778375 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.778378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.779523 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.779530 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.780285 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.780289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.780388 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.780411 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:09:23.780516 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 05:09:23.780587 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":3,"thread_start_us":91,"threads_started":1}
I20260807 05:09:23.780845 10551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.777001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:44875 (local address 127.8.16.65:45609)
0807 05:09:23.777138 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:23.777143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:23.777156 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:09:23.777204 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:23.777207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:23.777249 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:23.777314 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:23.777618 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.778088 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.779654 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.780288 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.780384 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.780443 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:23.780635 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 05:09:23.780711 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 05:09:23.781148 10457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ce8123e4d0489eb5cb989db4279c63" candidate_uuid: "30adf7aa145c48bb977af86f3c170bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9a9965cdf0f423b8663f311781b2390" is_pre_election: true
I20260807 05:09:23.781299 10457 raft_consensus.cc:2454] T b6ce8123e4d0489eb5cb989db4279c63 P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30adf7aa145c48bb977af86f3c170bef in term 0.
I20260807 05:09:23.781514 10584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.776404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:45609 (local address 127.8.16.66:44875)
0807 05:09:23.776714 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:23.776729 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:23.776827 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:23.777359 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:23.777362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:23.777379 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:23.777513 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.777520 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.778187 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.778190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.779510 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.779521 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.780414 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.780417 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.780507 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.780528 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 05:09:23.781280 (+ 752us) client_negotiation.cc:240] Negotiation successful
0807 05:09:23.781359 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":86,"threads_started":1}
I20260807 05:09:23.781685 10245 leader_election.cc:302] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30adf7aa145c48bb977af86f3c170bef, d9a9965cdf0f423b8663f311781b2390; no voters:
I20260807 05:09:23.781956 10535 raft_consensus.cc:2781] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:09:23.782027 10535 raft_consensus.cc:482] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:09:23.781989 10171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ce8123e4d0489eb5cb989db4279c63" candidate_uuid: "30adf7aa145c48bb977af86f3c170bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7985da867e914492888a84a9ae418d56" is_pre_election: true
I20260807 05:09:23.782064 10535 raft_consensus.cc:3037] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:23.782123 10171 raft_consensus.cc:2454] T b6ce8123e4d0489eb5cb989db4279c63 P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30adf7aa145c48bb977af86f3c170bef in term 0.
I20260807 05:09:23.783475 10535 raft_consensus.cc:504] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:23.783810 10535 leader_election.cc:288] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef [CANDIDATE]: Term 1 election: Requested vote from peers 7985da867e914492888a84a9ae418d56 (127.8.16.65:45609), d9a9965cdf0f423b8663f311781b2390 (127.8.16.67:37201)
I20260807 05:09:23.784096 10171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ce8123e4d0489eb5cb989db4279c63" candidate_uuid: "30adf7aa145c48bb977af86f3c170bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7985da867e914492888a84a9ae418d56"
I20260807 05:09:23.784123 10457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ce8123e4d0489eb5cb989db4279c63" candidate_uuid: "30adf7aa145c48bb977af86f3c170bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9a9965cdf0f423b8663f311781b2390"
I20260807 05:09:23.784219 10171 raft_consensus.cc:3037] T b6ce8123e4d0489eb5cb989db4279c63 P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:23.784248 10457 raft_consensus.cc:3037] T b6ce8123e4d0489eb5cb989db4279c63 P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:23.785120 10457 raft_consensus.cc:2454] T b6ce8123e4d0489eb5cb989db4279c63 P d9a9965cdf0f423b8663f311781b2390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30adf7aa145c48bb977af86f3c170bef in term 1.
I20260807 05:09:23.785285 10171 raft_consensus.cc:2454] T b6ce8123e4d0489eb5cb989db4279c63 P 7985da867e914492888a84a9ae418d56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30adf7aa145c48bb977af86f3c170bef in term 1.
I20260807 05:09:23.785393 10245 leader_election.cc:302] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30adf7aa145c48bb977af86f3c170bef, d9a9965cdf0f423b8663f311781b2390; no voters:
I20260807 05:09:23.785584 10535 raft_consensus.cc:2781] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:23.785812 10535 raft_consensus.cc:686] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef [term 1 LEADER]: Becoming Leader. State: Replica: 30adf7aa145c48bb977af86f3c170bef, State: Running, Role: LEADER
I20260807 05:09:23.786056 10535 consensus_queue.cc:237] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:23.788635 10012 catalog_manager.cc:5495] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef reported cstate change: term changed from 0 to 1, leader changed from <none> to 30adf7aa145c48bb977af86f3c170bef (127.8.16.66). New cstate: current_term: 1 leader_uuid: "30adf7aa145c48bb977af86f3c170bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } health_report { overall_health: UNKNOWN } } }
I20260807 05:09:23.811836 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.808448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37040 (local address 127.8.16.66:38383)
0807 05:09:23.808554 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:23.808559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:23.808574 (+ 15us) server_negotiation.cc:407] Connection header received
0807 05:09:23.808754 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:23.808756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:23.808807 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:23.808892 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:23.809119 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.809716 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.810506 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.810720 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.810807 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.811054 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:23.811485 (+ 431us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:23.811585 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:23.811662 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 05:09:23.811712 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 05:09:23.825330 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.822409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:54737 (local address 127.8.16.67:37201)
0807 05:09:23.822533 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:23.822538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:23.822553 (+ 15us) server_negotiation.cc:407] Connection header received
0807 05:09:23.822609 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:23.822612 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:23.822661 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:23.822741 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:23.822970 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.823467 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.824311 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.824531 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.824616 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.824897 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:23.824968 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:23.825026 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:23.825130 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 05:09:23.825181 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 05:09:23.825350 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.822100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:37201 (local address 127.8.16.66:54737)
0807 05:09:23.822402 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:23.822415 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:23.822508 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:23.822754 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:23.822756 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:23.822761 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 05:09:23.822860 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.822864 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.823569 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.823573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.824192 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.824201 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.824629 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.824632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.824718 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.824768 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 05:09:23.825044 (+ 276us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 05:09:23.825051 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 05:09:23.825110 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 05:09:23.825187 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":103,"threads_started":1}
I20260807 05:09:23.831959 10457 raft_consensus.cc:1261] T b6ce8123e4d0489eb5cb989db4279c63 P d9a9965cdf0f423b8663f311781b2390 [term 1 FOLLOWER]: Refusing update from remote peer 30adf7aa145c48bb977af86f3c170bef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:23.831959 10171 raft_consensus.cc:1261] T b6ce8123e4d0489eb5cb989db4279c63 P 7985da867e914492888a84a9ae418d56 [term 1 FOLLOWER]: Refusing update from remote peer 30adf7aa145c48bb977af86f3c170bef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:23.832432 10586 consensus_queue.cc:1035] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:23.832808 10586 consensus_queue.cc:1035] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:23.855278 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.852269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:48115 (local address 127.8.16.66:38383)
0807 05:09:23.852369 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:23.852374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:23.852388 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:09:23.852529 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:23.852532 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:23.852582 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:23.852662 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:23.852929 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.853531 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.854320 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.854536 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.854619 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.854846 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:23.854931 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:23.854988 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:23.855089 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 05:09:23.855137 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 05:09:23.855278 10575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:23.852206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:38383 (local address 127.8.16.67:48115)
0807 05:09:23.852331 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:23.852346 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:23.852420 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:23.852690 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:23.852693 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:23.852701 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 05:09:23.852803 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.852810 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.853625 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.853628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.854213 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:23.854221 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:23.854626 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:23.854629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:23.854714 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:23.854748 (+ 34us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 05:09:23.855017 (+ 269us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 05:09:23.855024 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 05:09:23.855081 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 05:09:23.855168 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 05:09:23.856254 10559 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 05:09:25.425882 10011 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:25.426045 10011 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:25.428325 10011 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e: Sending DeleteTablet for 3 replicas of tablet b6ce8123e4d0489eb5cb989db4279c63
I20260807 05:09:25.428921 10294 tablet_service.cc:1514] Processing DeleteTablet for tablet b6ce8123e4d0489eb5cb989db4279c63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:25 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:37026
I20260807 05:09:25.429050 10150 tablet_service.cc:1514] Processing DeleteTablet for tablet b6ce8123e4d0489eb5cb989db4279c63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:25 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:40954
I20260807 05:09:25.429050 10436 tablet_service.cc:1514] Processing DeleteTablet for tablet b6ce8123e4d0489eb5cb989db4279c63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:25 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:54670
I20260807 05:09:25.429411 10620 tablet_replica.cc:330] T b6ce8123e4d0489eb5cb989db4279c63 P d9a9965cdf0f423b8663f311781b2390: stopping tablet replica
I20260807 05:09:25.429432 10619 tablet_replica.cc:330] T b6ce8123e4d0489eb5cb989db4279c63 P 7985da867e914492888a84a9ae418d56: stopping tablet replica
I20260807 05:09:25.429563 10618 tablet_replica.cc:330] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef: stopping tablet replica
I20260807 05:09:25.429579 10619 raft_consensus.cc:2229] T b6ce8123e4d0489eb5cb989db4279c63 P 7985da867e914492888a84a9ae418d56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:25.429700 10619 raft_consensus.cc:2258] T b6ce8123e4d0489eb5cb989db4279c63 P 7985da867e914492888a84a9ae418d56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:25.429699 10620 raft_consensus.cc:2229] T b6ce8123e4d0489eb5cb989db4279c63 P d9a9965cdf0f423b8663f311781b2390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:25.429720 10618 raft_consensus.cc:2229] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:09:25.429850 10620 raft_consensus.cc:2258] T b6ce8123e4d0489eb5cb989db4279c63 P d9a9965cdf0f423b8663f311781b2390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:25.430032 10618 raft_consensus.cc:2258] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:25.430552 10620 ts_tablet_manager.cc:1905] T b6ce8123e4d0489eb5cb989db4279c63 P d9a9965cdf0f423b8663f311781b2390: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:25.430603 10618 ts_tablet_manager.cc:1905] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:25.433156 10619 ts_tablet_manager.cc:1905] T b6ce8123e4d0489eb5cb989db4279c63 P 7985da867e914492888a84a9ae418d56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:25.433199 10620 ts_tablet_manager.cc:1918] T b6ce8123e4d0489eb5cb989db4279c63 P d9a9965cdf0f423b8663f311781b2390: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 05:09:25.433260 10620 log.cc:1196] T b6ce8123e4d0489eb5cb989db4279c63 P d9a9965cdf0f423b8663f311781b2390: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/b6ce8123e4d0489eb5cb989db4279c63
I20260807 05:09:25.433310 10618 ts_tablet_manager.cc:1918] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 05:09:25.433383 10618 log.cc:1196] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-1/wal/wals/b6ce8123e4d0489eb5cb989db4279c63
I20260807 05:09:25.433576 10620 ts_tablet_manager.cc:1939] T b6ce8123e4d0489eb5cb989db4279c63 P d9a9965cdf0f423b8663f311781b2390: Deleting consensus metadata
I20260807 05:09:25.433669 10618 ts_tablet_manager.cc:1939] T b6ce8123e4d0489eb5cb989db4279c63 P 30adf7aa145c48bb977af86f3c170bef: Deleting consensus metadata
I20260807 05:09:25.434321 9996 catalog_manager.cc:4841] TS d9a9965cdf0f423b8663f311781b2390 (127.8.16.67:37201): tablet b6ce8123e4d0489eb5cb989db4279c63 (table test-table [id=ebb37b8fb91f4bfc9ee400d648b3849d]) successfully deleted
I20260807 05:09:25.434331 9997 catalog_manager.cc:4841] TS 30adf7aa145c48bb977af86f3c170bef (127.8.16.66:38383): tablet b6ce8123e4d0489eb5cb989db4279c63 (table test-table [id=ebb37b8fb91f4bfc9ee400d648b3849d]) successfully deleted
I20260807 05:09:25.435563 10619 ts_tablet_manager.cc:1918] T b6ce8123e4d0489eb5cb989db4279c63 P 7985da867e914492888a84a9ae418d56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 05:09:25.435626 10619 log.cc:1196] T b6ce8123e4d0489eb5cb989db4279c63 P 7985da867e914492888a84a9ae418d56: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-0/wal/wals/b6ce8123e4d0489eb5cb989db4279c63
I20260807 05:09:25.435837 10619 ts_tablet_manager.cc:1939] T b6ce8123e4d0489eb5cb989db4279c63 P 7985da867e914492888a84a9ae418d56: Deleting consensus metadata
I20260807 05:09:25.436518 9998 catalog_manager.cc:4841] TS 7985da867e914492888a84a9ae418d56 (127.8.16.65:45609): tablet b6ce8123e4d0489eb5cb989db4279c63 (table test-table [id=ebb37b8fb91f4bfc9ee400d648b3849d]) successfully deleted
Aug 07 05:09:25 dist-test-slave-49h3 krb5kdc[9966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079365, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 05:09:25 dist-test-slave-49h3 krb5kdc[9966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079365, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:25.461166 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:25.448018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48686 (local address 127.8.16.126:35103)
0807 05:09:25.448291 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:25.448298 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:25.448312 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:09:25.448421 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:25.448424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:25.448465 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:25.448538 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:25.449150 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:25.449767 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:25.450553 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:25.450773 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:25.450844 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:25.458469 (+ 7625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:25.458472 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:25.458473 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:25.458499 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:25.459847 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:25.460198 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:25.460200 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:25.460202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:25.460272 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:25.460446 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:25.460448 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:25.460449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:25.460620 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:25.460718 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:25.460879 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 05:09:25.460980 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":94,"threads_started":1}
I20260807 05:09:25.463474 10011 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48686:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:09:25.463863 10011 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:09:25.468211 10294 tablet_service.cc:1467] Processing CreateTablet for tablet 6dfa70cf66f241ed9376a5d572a58443 (DEFAULT_TABLE table=test-table [id=dc0babf7b5b6425ca73a255ade75bcbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:25.468210 10150 tablet_service.cc:1467] Processing CreateTablet for tablet 6dfa70cf66f241ed9376a5d572a58443 (DEFAULT_TABLE table=test-table [id=dc0babf7b5b6425ca73a255ade75bcbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:25.468281 10436 tablet_service.cc:1467] Processing CreateTablet for tablet 6dfa70cf66f241ed9376a5d572a58443 (DEFAULT_TABLE table=test-table [id=dc0babf7b5b6425ca73a255ade75bcbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:25.468506 10150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dfa70cf66f241ed9376a5d572a58443. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:25.468509 10294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dfa70cf66f241ed9376a5d572a58443. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:25.468607 10436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dfa70cf66f241ed9376a5d572a58443. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:25.471189 10630 tablet_bootstrap.cc:492] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef: Bootstrap starting.
I20260807 05:09:25.471272 10629 tablet_bootstrap.cc:492] T 6dfa70cf66f241ed9376a5d572a58443 P d9a9965cdf0f423b8663f311781b2390: Bootstrap starting.
I20260807 05:09:25.471292 10631 tablet_bootstrap.cc:492] T 6dfa70cf66f241ed9376a5d572a58443 P 7985da867e914492888a84a9ae418d56: Bootstrap starting.
I20260807 05:09:25.472126 10631 tablet_bootstrap.cc:654] T 6dfa70cf66f241ed9376a5d572a58443 P 7985da867e914492888a84a9ae418d56: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:25.472129 10630 tablet_bootstrap.cc:654] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:25.472128 10629 tablet_bootstrap.cc:654] T 6dfa70cf66f241ed9376a5d572a58443 P d9a9965cdf0f423b8663f311781b2390: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:25.473188 10630 tablet_bootstrap.cc:492] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef: No bootstrap required, opened a new log
I20260807 05:09:25.473193 10629 tablet_bootstrap.cc:492] T 6dfa70cf66f241ed9376a5d572a58443 P d9a9965cdf0f423b8663f311781b2390: No bootstrap required, opened a new log
I20260807 05:09:25.473201 10631 tablet_bootstrap.cc:492] T 6dfa70cf66f241ed9376a5d572a58443 P 7985da867e914492888a84a9ae418d56: No bootstrap required, opened a new log
I20260807 05:09:25.473269 10631 ts_tablet_manager.cc:1397] T 6dfa70cf66f241ed9376a5d572a58443 P 7985da867e914492888a84a9ae418d56: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:25.473275 10630 ts_tablet_manager.cc:1397] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:25.473269 10629 ts_tablet_manager.cc:1397] T 6dfa70cf66f241ed9376a5d572a58443 P d9a9965cdf0f423b8663f311781b2390: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:25.473709 10631 raft_consensus.cc:348] T 6dfa70cf66f241ed9376a5d572a58443 P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:25.473738 10630 raft_consensus.cc:348] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:25.473821 10631 raft_consensus.cc:374] T 6dfa70cf66f241ed9376a5d572a58443 P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:25.473788 10629 raft_consensus.cc:348] T 6dfa70cf66f241ed9376a5d572a58443 P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:25.473851 10631 raft_consensus.cc:729] T 6dfa70cf66f241ed9376a5d572a58443 P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7985da867e914492888a84a9ae418d56, State: Initialized, Role: FOLLOWER
I20260807 05:09:25.473848 10630 raft_consensus.cc:374] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:25.473877 10629 raft_consensus.cc:374] T 6dfa70cf66f241ed9376a5d572a58443 P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:25.473886 10630 raft_consensus.cc:729] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30adf7aa145c48bb977af86f3c170bef, State: Initialized, Role: FOLLOWER
I20260807 05:09:25.473907 10629 raft_consensus.cc:729] T 6dfa70cf66f241ed9376a5d572a58443 P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9a9965cdf0f423b8663f311781b2390, State: Initialized, Role: FOLLOWER
I20260807 05:09:25.473975 10631 consensus_queue.cc:260] T 6dfa70cf66f241ed9376a5d572a58443 P 7985da867e914492888a84a9ae418d56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:25.474007 10629 consensus_queue.cc:260] T 6dfa70cf66f241ed9376a5d572a58443 P d9a9965cdf0f423b8663f311781b2390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:25.474007 10630 consensus_queue.cc:260] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:25.474257 10629 ts_tablet_manager.cc:1428] T 6dfa70cf66f241ed9376a5d572a58443 P d9a9965cdf0f423b8663f311781b2390: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:09:25.474316 10631 ts_tablet_manager.cc:1428] T 6dfa70cf66f241ed9376a5d572a58443 P 7985da867e914492888a84a9ae418d56: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:09:25.474895 10630 ts_tablet_manager.cc:1428] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:25.536770 10596 raft_consensus.cc:482] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:09:25.536908 10596 raft_consensus.cc:504] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:25.537335 10596 leader_election.cc:288] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7985da867e914492888a84a9ae418d56 (127.8.16.65:45609), d9a9965cdf0f423b8663f311781b2390 (127.8.16.67:37201)
I20260807 05:09:25.537788 10171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dfa70cf66f241ed9376a5d572a58443" candidate_uuid: "30adf7aa145c48bb977af86f3c170bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7985da867e914492888a84a9ae418d56" is_pre_election: true
I20260807 05:09:25.537799 10457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dfa70cf66f241ed9376a5d572a58443" candidate_uuid: "30adf7aa145c48bb977af86f3c170bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9a9965cdf0f423b8663f311781b2390" is_pre_election: true
I20260807 05:09:25.537930 10457 raft_consensus.cc:2454] T 6dfa70cf66f241ed9376a5d572a58443 P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30adf7aa145c48bb977af86f3c170bef in term 0.
I20260807 05:09:25.537943 10171 raft_consensus.cc:2454] T 6dfa70cf66f241ed9376a5d572a58443 P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30adf7aa145c48bb977af86f3c170bef in term 0.
I20260807 05:09:25.538264 10245 leader_election.cc:302] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30adf7aa145c48bb977af86f3c170bef, d9a9965cdf0f423b8663f311781b2390; no voters:
I20260807 05:09:25.538435 10596 raft_consensus.cc:2781] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:09:25.538497 10596 raft_consensus.cc:482] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:09:25.538533 10596 raft_consensus.cc:3037] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:25.539570 10596 raft_consensus.cc:504] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:25.539865 10596 leader_election.cc:288] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef [CANDIDATE]: Term 1 election: Requested vote from peers 7985da867e914492888a84a9ae418d56 (127.8.16.65:45609), d9a9965cdf0f423b8663f311781b2390 (127.8.16.67:37201)
I20260807 05:09:25.540207 10171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dfa70cf66f241ed9376a5d572a58443" candidate_uuid: "30adf7aa145c48bb977af86f3c170bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7985da867e914492888a84a9ae418d56"
I20260807 05:09:25.540246 10457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dfa70cf66f241ed9376a5d572a58443" candidate_uuid: "30adf7aa145c48bb977af86f3c170bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9a9965cdf0f423b8663f311781b2390"
I20260807 05:09:25.540321 10171 raft_consensus.cc:3037] T 6dfa70cf66f241ed9376a5d572a58443 P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:25.540356 10457 raft_consensus.cc:3037] T 6dfa70cf66f241ed9376a5d572a58443 P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:25.541222 10171 raft_consensus.cc:2454] T 6dfa70cf66f241ed9376a5d572a58443 P 7985da867e914492888a84a9ae418d56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30adf7aa145c48bb977af86f3c170bef in term 1.
I20260807 05:09:25.541311 10457 raft_consensus.cc:2454] T 6dfa70cf66f241ed9376a5d572a58443 P d9a9965cdf0f423b8663f311781b2390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30adf7aa145c48bb977af86f3c170bef in term 1.
I20260807 05:09:25.541519 10247 leader_election.cc:302] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30adf7aa145c48bb977af86f3c170bef, 7985da867e914492888a84a9ae418d56; no voters:
I20260807 05:09:25.541756 10596 raft_consensus.cc:2781] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:25.541842 10596 raft_consensus.cc:686] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef [term 1 LEADER]: Becoming Leader. State: Replica: 30adf7aa145c48bb977af86f3c170bef, State: Running, Role: LEADER
I20260807 05:09:25.541978 10596 consensus_queue.cc:237] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:25.543596 10012 catalog_manager.cc:5495] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef reported cstate change: term changed from 0 to 1, leader changed from <none> to 30adf7aa145c48bb977af86f3c170bef (127.8.16.66). New cstate: current_term: 1 leader_uuid: "30adf7aa145c48bb977af86f3c170bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } health_report { overall_health: UNKNOWN } } }
I20260807 05:09:25.557114 10638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:25.553985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37048 (local address 127.8.16.66:38383)
0807 05:09:25.554270 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:25.554277 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:25.554287 (+ 10us) server_negotiation.cc:407] Connection header received
0807 05:09:25.554321 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:25.554323 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:25.554362 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:25.554433 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:25.554677 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:25.555251 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:25.556111 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:25.556374 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:25.556459 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:25.556707 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:25.556776 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:25.556831 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:25.556921 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 05:09:25.556971 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":122,"threads_started":1}
I20260807 05:09:25.562431 10457 raft_consensus.cc:1261] T 6dfa70cf66f241ed9376a5d572a58443 P d9a9965cdf0f423b8663f311781b2390 [term 1 FOLLOWER]: Refusing update from remote peer 30adf7aa145c48bb977af86f3c170bef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:25.562525 10171 raft_consensus.cc:1261] T 6dfa70cf66f241ed9376a5d572a58443 P 7985da867e914492888a84a9ae418d56 [term 1 FOLLOWER]: Refusing update from remote peer 30adf7aa145c48bb977af86f3c170bef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:25.562759 10609 consensus_queue.cc:1035] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:25.562934 10596 consensus_queue.cc:1035] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:25.565892 10601 mvcc.cc:205] Tried to move back new op lower bound from 7315781081340276736 to 7315781081261129728. Current Snapshot: MvccSnapshot[applied={T|T < 7315781081340276736}]
I20260807 05:09:25.591930 10011 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:25.592072 10011 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:25.593847 10011 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e: Sending DeleteTablet for 3 replicas of tablet 6dfa70cf66f241ed9376a5d572a58443
I20260807 05:09:25.594365 10294 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dfa70cf66f241ed9376a5d572a58443 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:25 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:37026
I20260807 05:09:25.594434 10436 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dfa70cf66f241ed9376a5d572a58443 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:25 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:54670
I20260807 05:09:25.594441 10150 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dfa70cf66f241ed9376a5d572a58443 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:25 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:40954
I20260807 05:09:25.594596 10620 tablet_replica.cc:330] T 6dfa70cf66f241ed9376a5d572a58443 P d9a9965cdf0f423b8663f311781b2390: stopping tablet replica
I20260807 05:09:25.594623 10618 tablet_replica.cc:330] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef: stopping tablet replica
I20260807 05:09:25.594705 10620 raft_consensus.cc:2229] T 6dfa70cf66f241ed9376a5d572a58443 P d9a9965cdf0f423b8663f311781b2390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:25.594722 10618 raft_consensus.cc:2229] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:09:25.594801 10620 raft_consensus.cc:2258] T 6dfa70cf66f241ed9376a5d572a58443 P d9a9965cdf0f423b8663f311781b2390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:25.594846 10618 raft_consensus.cc:2258] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:25.594996 10619 tablet_replica.cc:330] T 6dfa70cf66f241ed9376a5d572a58443 P 7985da867e914492888a84a9ae418d56: stopping tablet replica
I20260807 05:09:25.595108 10619 raft_consensus.cc:2229] T 6dfa70cf66f241ed9376a5d572a58443 P 7985da867e914492888a84a9ae418d56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:25.595204 10619 raft_consensus.cc:2258] T 6dfa70cf66f241ed9376a5d572a58443 P 7985da867e914492888a84a9ae418d56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:25.595212 10011 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48686:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 05:09:25.595402 10620 ts_tablet_manager.cc:1905] T 6dfa70cf66f241ed9376a5d572a58443 P d9a9965cdf0f423b8663f311781b2390: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 05:09:25.595577 10011 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:09:25.596032 10619 ts_tablet_manager.cc:1905] T 6dfa70cf66f241ed9376a5d572a58443 P 7985da867e914492888a84a9ae418d56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:25.597723 10620 ts_tablet_manager.cc:1918] T 6dfa70cf66f241ed9376a5d572a58443 P d9a9965cdf0f423b8663f311781b2390: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:09:25.597796 10620 log.cc:1196] T 6dfa70cf66f241ed9376a5d572a58443 P d9a9965cdf0f423b8663f311781b2390: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/6dfa70cf66f241ed9376a5d572a58443
I20260807 05:09:25.597977 10619 ts_tablet_manager.cc:1918] T 6dfa70cf66f241ed9376a5d572a58443 P 7985da867e914492888a84a9ae418d56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:09:25.598022 10620 ts_tablet_manager.cc:1939] T 6dfa70cf66f241ed9376a5d572a58443 P d9a9965cdf0f423b8663f311781b2390: Deleting consensus metadata
I20260807 05:09:25.598039 10619 log.cc:1196] T 6dfa70cf66f241ed9376a5d572a58443 P 7985da867e914492888a84a9ae418d56: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-0/wal/wals/6dfa70cf66f241ed9376a5d572a58443
I20260807 05:09:25.598237 10619 ts_tablet_manager.cc:1939] T 6dfa70cf66f241ed9376a5d572a58443 P 7985da867e914492888a84a9ae418d56: Deleting consensus metadata
I20260807 05:09:25.598667 9996 catalog_manager.cc:4841] TS d9a9965cdf0f423b8663f311781b2390 (127.8.16.67:37201): tablet 6dfa70cf66f241ed9376a5d572a58443 (table test-table [id=dc0babf7b5b6425ca73a255ade75bcbd]) successfully deleted
I20260807 05:09:25.598809 10618 ts_tablet_manager.cc:1905] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:25.599030 9998 catalog_manager.cc:4841] TS 7985da867e914492888a84a9ae418d56 (127.8.16.65:45609): tablet 6dfa70cf66f241ed9376a5d572a58443 (table test-table [id=dc0babf7b5b6425ca73a255ade75bcbd]) successfully deleted
I20260807 05:09:25.601150 10618 ts_tablet_manager.cc:1918] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:09:25.601222 10618 log.cc:1196] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-1/wal/wals/6dfa70cf66f241ed9376a5d572a58443
I20260807 05:09:25.601428 10618 ts_tablet_manager.cc:1939] T 6dfa70cf66f241ed9376a5d572a58443 P 30adf7aa145c48bb977af86f3c170bef: Deleting consensus metadata
I20260807 05:09:25.601711 10150 tablet_service.cc:1467] Processing CreateTablet for tablet 3f520ed87e554311bf8df4a86d09e6fa (DEFAULT_TABLE table=test-table [id=222eef76815f4b2c9c6ad70f91f9c08f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:25.601949 10150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f520ed87e554311bf8df4a86d09e6fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:25.603806 10631 tablet_bootstrap.cc:492] T 3f520ed87e554311bf8df4a86d09e6fa P 7985da867e914492888a84a9ae418d56: Bootstrap starting.
I20260807 05:09:25.603910 9997 catalog_manager.cc:4841] TS 30adf7aa145c48bb977af86f3c170bef (127.8.16.66:38383): tablet 6dfa70cf66f241ed9376a5d572a58443 (table test-table [id=dc0babf7b5b6425ca73a255ade75bcbd]) successfully deleted
I20260807 05:09:25.604194 10294 tablet_service.cc:1467] Processing CreateTablet for tablet 3f520ed87e554311bf8df4a86d09e6fa (DEFAULT_TABLE table=test-table [id=222eef76815f4b2c9c6ad70f91f9c08f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:25.604262 10436 tablet_service.cc:1467] Processing CreateTablet for tablet 3f520ed87e554311bf8df4a86d09e6fa (DEFAULT_TABLE table=test-table [id=222eef76815f4b2c9c6ad70f91f9c08f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:25.604440 10294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f520ed87e554311bf8df4a86d09e6fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:25.604485 10436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f520ed87e554311bf8df4a86d09e6fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:25.604532 10631 tablet_bootstrap.cc:654] T 3f520ed87e554311bf8df4a86d09e6fa P 7985da867e914492888a84a9ae418d56: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:25.605496 10631 tablet_bootstrap.cc:492] T 3f520ed87e554311bf8df4a86d09e6fa P 7985da867e914492888a84a9ae418d56: No bootstrap required, opened a new log
I20260807 05:09:25.605576 10631 ts_tablet_manager.cc:1397] T 3f520ed87e554311bf8df4a86d09e6fa P 7985da867e914492888a84a9ae418d56: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:25.605988 10631 raft_consensus.cc:348] T 3f520ed87e554311bf8df4a86d09e6fa P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:25.606101 10631 raft_consensus.cc:374] T 3f520ed87e554311bf8df4a86d09e6fa P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:25.606123 10631 raft_consensus.cc:729] T 3f520ed87e554311bf8df4a86d09e6fa P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7985da867e914492888a84a9ae418d56, State: Initialized, Role: FOLLOWER
I20260807 05:09:25.606227 10631 consensus_queue.cc:260] T 3f520ed87e554311bf8df4a86d09e6fa P 7985da867e914492888a84a9ae418d56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:25.606468 10631 ts_tablet_manager.cc:1428] T 3f520ed87e554311bf8df4a86d09e6fa P 7985da867e914492888a84a9ae418d56: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 05:09:25.606703 10629 tablet_bootstrap.cc:492] T 3f520ed87e554311bf8df4a86d09e6fa P d9a9965cdf0f423b8663f311781b2390: Bootstrap starting.
I20260807 05:09:25.607479 10629 tablet_bootstrap.cc:654] T 3f520ed87e554311bf8df4a86d09e6fa P d9a9965cdf0f423b8663f311781b2390: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:25.607998 10630 tablet_bootstrap.cc:492] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef: Bootstrap starting.
I20260807 05:09:25.608805 10630 tablet_bootstrap.cc:654] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:25.609740 10629 tablet_bootstrap.cc:492] T 3f520ed87e554311bf8df4a86d09e6fa P d9a9965cdf0f423b8663f311781b2390: No bootstrap required, opened a new log
I20260807 05:09:25.609825 10629 ts_tablet_manager.cc:1397] T 3f520ed87e554311bf8df4a86d09e6fa P d9a9965cdf0f423b8663f311781b2390: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 05:09:25.610178 10629 raft_consensus.cc:348] T 3f520ed87e554311bf8df4a86d09e6fa P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:25.610289 10629 raft_consensus.cc:374] T 3f520ed87e554311bf8df4a86d09e6fa P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:25.610322 10629 raft_consensus.cc:729] T 3f520ed87e554311bf8df4a86d09e6fa P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9a9965cdf0f423b8663f311781b2390, State: Initialized, Role: FOLLOWER
I20260807 05:09:25.610438 10629 consensus_queue.cc:260] T 3f520ed87e554311bf8df4a86d09e6fa P d9a9965cdf0f423b8663f311781b2390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:25.610700 10629 ts_tablet_manager.cc:1428] T 3f520ed87e554311bf8df4a86d09e6fa P d9a9965cdf0f423b8663f311781b2390: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 05:09:25.611196 10630 tablet_bootstrap.cc:492] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef: No bootstrap required, opened a new log
I20260807 05:09:25.611281 10630 ts_tablet_manager.cc:1397] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 05:09:25.611632 10630 raft_consensus.cc:348] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:25.611734 10630 raft_consensus.cc:374] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:25.611760 10630 raft_consensus.cc:729] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30adf7aa145c48bb977af86f3c170bef, State: Initialized, Role: FOLLOWER
I20260807 05:09:25.611864 10630 consensus_queue.cc:260] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:25.612084 10630 ts_tablet_manager.cc:1428] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:09:25.644178 10609 raft_consensus.cc:482] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:09:25.644321 10609 raft_consensus.cc:504] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:25.644735 10609 leader_election.cc:288] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7985da867e914492888a84a9ae418d56 (127.8.16.65:45609), d9a9965cdf0f423b8663f311781b2390 (127.8.16.67:37201)
I20260807 05:09:25.645124 10171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f520ed87e554311bf8df4a86d09e6fa" candidate_uuid: "30adf7aa145c48bb977af86f3c170bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7985da867e914492888a84a9ae418d56" is_pre_election: true
I20260807 05:09:25.645143 10457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f520ed87e554311bf8df4a86d09e6fa" candidate_uuid: "30adf7aa145c48bb977af86f3c170bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9a9965cdf0f423b8663f311781b2390" is_pre_election: true
I20260807 05:09:25.645272 10171 raft_consensus.cc:2454] T 3f520ed87e554311bf8df4a86d09e6fa P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30adf7aa145c48bb977af86f3c170bef in term 0.
I20260807 05:09:25.645285 10457 raft_consensus.cc:2454] T 3f520ed87e554311bf8df4a86d09e6fa P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30adf7aa145c48bb977af86f3c170bef in term 0.
I20260807 05:09:25.645584 10245 leader_election.cc:302] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 30adf7aa145c48bb977af86f3c170bef, 7985da867e914492888a84a9ae418d56, d9a9965cdf0f423b8663f311781b2390; no voters:
I20260807 05:09:25.645766 10609 raft_consensus.cc:2781] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:09:25.645826 10609 raft_consensus.cc:482] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:09:25.645851 10609 raft_consensus.cc:3037] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:25.646974 10609 raft_consensus.cc:504] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:25.647289 10609 leader_election.cc:288] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef [CANDIDATE]: Term 1 election: Requested vote from peers 7985da867e914492888a84a9ae418d56 (127.8.16.65:45609), d9a9965cdf0f423b8663f311781b2390 (127.8.16.67:37201)
I20260807 05:09:25.647620 10457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f520ed87e554311bf8df4a86d09e6fa" candidate_uuid: "30adf7aa145c48bb977af86f3c170bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9a9965cdf0f423b8663f311781b2390"
I20260807 05:09:25.647614 10171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f520ed87e554311bf8df4a86d09e6fa" candidate_uuid: "30adf7aa145c48bb977af86f3c170bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7985da867e914492888a84a9ae418d56"
I20260807 05:09:25.647733 10171 raft_consensus.cc:3037] T 3f520ed87e554311bf8df4a86d09e6fa P 7985da867e914492888a84a9ae418d56 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:25.647742 10457 raft_consensus.cc:3037] T 3f520ed87e554311bf8df4a86d09e6fa P d9a9965cdf0f423b8663f311781b2390 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:25.648618 10171 raft_consensus.cc:2454] T 3f520ed87e554311bf8df4a86d09e6fa P 7985da867e914492888a84a9ae418d56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30adf7aa145c48bb977af86f3c170bef in term 1.
I20260807 05:09:25.648618 10457 raft_consensus.cc:2454] T 3f520ed87e554311bf8df4a86d09e6fa P d9a9965cdf0f423b8663f311781b2390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30adf7aa145c48bb977af86f3c170bef in term 1.
I20260807 05:09:25.648895 10247 leader_election.cc:302] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30adf7aa145c48bb977af86f3c170bef, 7985da867e914492888a84a9ae418d56; no voters:
I20260807 05:09:25.649091 10609 raft_consensus.cc:2781] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:25.649168 10609 raft_consensus.cc:686] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef [term 1 LEADER]: Becoming Leader. State: Replica: 30adf7aa145c48bb977af86f3c170bef, State: Running, Role: LEADER
I20260807 05:09:25.649300 10609 consensus_queue.cc:237] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } }
I20260807 05:09:25.650791 10012 catalog_manager.cc:5495] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef reported cstate change: term changed from 0 to 1, leader changed from <none> to 30adf7aa145c48bb977af86f3c170bef (127.8.16.66). New cstate: current_term: 1 leader_uuid: "30adf7aa145c48bb977af86f3c170bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30adf7aa145c48bb977af86f3c170bef" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 38383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 } health_report { overall_health: UNKNOWN } } }
I20260807 05:09:25.664510 10171 raft_consensus.cc:1261] T 3f520ed87e554311bf8df4a86d09e6fa P 7985da867e914492888a84a9ae418d56 [term 1 FOLLOWER]: Refusing update from remote peer 30adf7aa145c48bb977af86f3c170bef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:25.664606 10457 raft_consensus.cc:1261] T 3f520ed87e554311bf8df4a86d09e6fa P d9a9965cdf0f423b8663f311781b2390 [term 1 FOLLOWER]: Refusing update from remote peer 30adf7aa145c48bb977af86f3c170bef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:25.664903 10609 consensus_queue.cc:1035] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "7985da867e914492888a84a9ae418d56" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:25.665062 10642 consensus_queue.cc:1035] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9a9965cdf0f423b8663f311781b2390" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 37201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 05:09:25.676520 10600 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 05:09:27.222793 10012 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:27.222971 10012 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:27.225328 10012 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab5ceab05697435281630d85fe06610e: Sending DeleteTablet for 3 replicas of tablet 3f520ed87e554311bf8df4a86d09e6fa
I20260807 05:09:27.225883 10150 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f520ed87e554311bf8df4a86d09e6fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:27 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:40954
I20260807 05:09:27.225880 10294 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f520ed87e554311bf8df4a86d09e6fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:27 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:37026
I20260807 05:09:27.225970 10435 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f520ed87e554311bf8df4a86d09e6fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:27 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:54670
I20260807 05:09:27.226261 10671 tablet_replica.cc:330] T 3f520ed87e554311bf8df4a86d09e6fa P 7985da867e914492888a84a9ae418d56: stopping tablet replica
I20260807 05:09:27.226361 10673 tablet_replica.cc:330] T 3f520ed87e554311bf8df4a86d09e6fa P d9a9965cdf0f423b8663f311781b2390: stopping tablet replica
I20260807 05:09:27.226418 10672 tablet_replica.cc:330] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef: stopping tablet replica
I20260807 05:09:27.226428 10671 raft_consensus.cc:2229] T 3f520ed87e554311bf8df4a86d09e6fa P 7985da867e914492888a84a9ae418d56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:27.226540 10671 raft_consensus.cc:2258] T 3f520ed87e554311bf8df4a86d09e6fa P 7985da867e914492888a84a9ae418d56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:27.226555 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 10084
I20260807 05:09:27.226568 10672 raft_consensus.cc:2229] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:09:27.229532 10672 raft_consensus.cc:2258] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:27.226612 10673 raft_consensus.cc:2229] T 3f520ed87e554311bf8df4a86d09e6fa P d9a9965cdf0f423b8663f311781b2390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:27.230132 10673 raft_consensus.cc:2258] T 3f520ed87e554311bf8df4a86d09e6fa P d9a9965cdf0f423b8663f311781b2390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:27.230718 10673 ts_tablet_manager.cc:1905] T 3f520ed87e554311bf8df4a86d09e6fa P d9a9965cdf0f423b8663f311781b2390: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:27.232823 10673 ts_tablet_manager.cc:1918] T 3f520ed87e554311bf8df4a86d09e6fa P d9a9965cdf0f423b8663f311781b2390: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 05:09:27.232878 10673 log.cc:1196] T 3f520ed87e554311bf8df4a86d09e6fa P d9a9965cdf0f423b8663f311781b2390: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/3f520ed87e554311bf8df4a86d09e6fa
I20260807 05:09:27.233052 10673 ts_tablet_manager.cc:1939] T 3f520ed87e554311bf8df4a86d09e6fa P d9a9965cdf0f423b8663f311781b2390: Deleting consensus metadata
I20260807 05:09:27.233916 9996 catalog_manager.cc:4841] TS d9a9965cdf0f423b8663f311781b2390 (127.8.16.67:37201): tablet 3f520ed87e554311bf8df4a86d09e6fa (table test-table [id=222eef76815f4b2c9c6ad70f91f9c08f]) successfully deleted
I20260807 05:09:27.234289 10672 ts_tablet_manager.cc:1905] T 3f520ed87e554311bf8df4a86d09e6fa P 30adf7aa145c48bb977af86f3c170bef: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 05:09:27.236049 9998 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.16.65:45609 (error 108)
I20260807 05:09:27.236438 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 10227
I20260807 05:09:27.239053 10675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:27.238160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:45609 (local address 127.0.0.1:48180)
0807 05:09:27.238892 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:27.238962 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.65:45609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":129,"threads_started":1}
W20260807 05:09:27.239228 9998 catalog_manager.cc:4584] TS 7985da867e914492888a84a9ae418d56 (127.8.16.65:45609): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3f520ed87e554311bf8df4a86d09e6fa: Network error: Client connection negotiation failed: client connection to 127.8.16.65:45609: connect: Connection refused (error 111)
I20260807 05:09:27.243580 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 10371
I20260807 05:09:27.244531 10675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:27.243615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:38383 (local address 127.0.0.1:41348)
0807 05:09:27.244411 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:27.244459 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.66:38383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 05:09:27.251192 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 9979
2026-08-07T05:09:27Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6393 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:09:27 dist-test-slave-49h3 krb5kdc[10678](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:09:27 dist-test-slave-49h3 krb5kdc[10678](info): set up 2 sockets
Aug 07 05:09:27 dist-test-slave-49h3 krb5kdc[10678](info): commencing operation
krb5kdc: starting...
W20260807 05:09:28.241111 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.966s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:09:28 dist-test-slave-49h3 krb5kdc[10678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:09:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:09:28.360006 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:45143
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:40355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:45143
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:28.467109 10691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:28.467386 10691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:28.467425 10691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:28.471416 10691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:28.471479 10691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:28.471498 10691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:28.471513 10691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:28.471530 10691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:28.476799 10691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:40355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:45143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:45143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:28.477856 10691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:28.478839 10691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:28.486819 10697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:28.486840 10699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:28.486819 10696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:28.486986 10691 server_base.cc:1034] running on GCE node
I20260807 05:09:28.487646 10691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:28.488655 10691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:28.489800 10691 hybrid_clock.cc:648] HybridClock initialized: now 1786079368489781 us; error 34 us; skew 500 ppm
Aug 07 05:09:28 dist-test-slave-49h3 krb5kdc[10678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079368, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:28.492406 10691 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:09:28.493247 10691 webserver.cc:460] Webserver started at http://127.8.16.126:32865/ using document root <none> and password file <none>
I20260807 05:09:28.493784 10691 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:28.493850 10691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:28.494042 10691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:28.495962 10691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "8eb286130fd64c8a85965ef420fa8834"
format_stamp: "Formatted at 2026-08-07 05:09:28 on dist-test-slave-49h3"
server_key: "c10e516de7608a6d93022eb2eafd003b"
server_key_iv: "163d098ea8a1e941b1c633f5be17d23d"
server_key_version: "encryptionkey@0"
I20260807 05:09:28.496487 10691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "8eb286130fd64c8a85965ef420fa8834"
format_stamp: "Formatted at 2026-08-07 05:09:28 on dist-test-slave-49h3"
server_key: "c10e516de7608a6d93022eb2eafd003b"
server_key_iv: "163d098ea8a1e941b1c633f5be17d23d"
server_key_version: "encryptionkey@0"
I20260807 05:09:28.499923 10691 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:09:28.502152 10706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:28.503438 10691 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:28.503556 10691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "8eb286130fd64c8a85965ef420fa8834"
format_stamp: "Formatted at 2026-08-07 05:09:28 on dist-test-slave-49h3"
server_key: "c10e516de7608a6d93022eb2eafd003b"
server_key_iv: "163d098ea8a1e941b1c633f5be17d23d"
server_key_version: "encryptionkey@0"
I20260807 05:09:28.503654 10691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:28.514680 10691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:28.517338 10691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:28.517531 10691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:28.523948 10691 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:45143
I20260807 05:09:28.523963 10760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:45143 every 8 connection(s)
I20260807 05:09:28.524857 10691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:28.526147 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 10691
I20260807 05:09:28.526265 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:09:28.526554 8257 external_mini_cluster.cc:1428] Setting key eb247b47cd4aa047b9280498c0d72a11
I20260807 05:09:28.527949 10761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:28.533970 10761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8eb286130fd64c8a85965ef420fa8834: Bootstrap starting.
Aug 07 05:09:28 dist-test-slave-49h3 krb5kdc[10678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:28.536441 10761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8eb286130fd64c8a85965ef420fa8834: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:28.537119 10761 log.cc:824] T 00000000000000000000000000000000 P 8eb286130fd64c8a85965ef420fa8834: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:28.539252 10761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8eb286130fd64c8a85965ef420fa8834: No bootstrap required, opened a new log
I20260807 05:09:28.540381 10764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:28.527875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46812 (local address 127.8.16.126:45143)
0807 05:09:28.528285 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:28.528301 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:28.528338 (+ 37us) server_negotiation.cc:407] Connection header received
0807 05:09:28.529012 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:28.529029 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:28.529346 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:28.529665 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:28.530525 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:28.531483 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:28.532586 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:28.533033 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:28.533141 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:28.535890 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:28.535896 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:28.535912 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:28.535952 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:28.537748 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:28.538209 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:28.538214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:28.538218 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:28.538325 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:28.538576 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:28.538578 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:28.538579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:28.538953 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:28.539090 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:28.539387 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 05:09:28.539686 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":139,"threads_started":1}
I20260807 05:09:28.543063 10761 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8eb286130fd64c8a85965ef420fa8834 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb286130fd64c8a85965ef420fa8834" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 45143 } }
I20260807 05:09:28.543285 10761 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8eb286130fd64c8a85965ef420fa8834 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:28.543340 10761 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8eb286130fd64c8a85965ef420fa8834 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb286130fd64c8a85965ef420fa8834, State: Initialized, Role: FOLLOWER
I20260807 05:09:28.543717 10761 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8eb286130fd64c8a85965ef420fa8834 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb286130fd64c8a85965ef420fa8834" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 45143 } }
I20260807 05:09:28.543857 10761 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8eb286130fd64c8a85965ef420fa8834 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:09:28.543946 10761 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8eb286130fd64c8a85965ef420fa8834 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:09:28.544054 10761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8eb286130fd64c8a85965ef420fa8834 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:28.544991 10761 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8eb286130fd64c8a85965ef420fa8834 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb286130fd64c8a85965ef420fa8834" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 45143 } }
I20260807 05:09:28.545307 10761 leader_election.cc:302] T 00000000000000000000000000000000 P 8eb286130fd64c8a85965ef420fa8834 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8eb286130fd64c8a85965ef420fa8834; no voters:
I20260807 05:09:28.545640 10761 leader_election.cc:288] T 00000000000000000000000000000000 P 8eb286130fd64c8a85965ef420fa8834 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:09:28.545773 10766 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8eb286130fd64c8a85965ef420fa8834 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:28.545981 10766 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8eb286130fd64c8a85965ef420fa8834 [term 1 LEADER]: Becoming Leader. State: Replica: 8eb286130fd64c8a85965ef420fa8834, State: Running, Role: LEADER
I20260807 05:09:28.546316 10766 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8eb286130fd64c8a85965ef420fa8834 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb286130fd64c8a85965ef420fa8834" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 45143 } }
I20260807 05:09:28.546789 10761 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8eb286130fd64c8a85965ef420fa8834 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:28.547832 10767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eb286130fd64c8a85965ef420fa8834 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8eb286130fd64c8a85965ef420fa8834" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb286130fd64c8a85965ef420fa8834" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 45143 } } }
I20260807 05:09:28.547968 10767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eb286130fd64c8a85965ef420fa8834 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:28.548308 10768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eb286130fd64c8a85965ef420fa8834 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8eb286130fd64c8a85965ef420fa8834. Latest consensus state: current_term: 1 leader_uuid: "8eb286130fd64c8a85965ef420fa8834" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb286130fd64c8a85965ef420fa8834" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 45143 } } }
I20260807 05:09:28.548362 10775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:28.548475 10768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eb286130fd64c8a85965ef420fa8834 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:28.551465 10775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:28.556792 10775 catalog_manager.cc:1347] Generated new cluster ID: 8f822de3e8524b1fabf30ec3dc9539d1
I20260807 05:09:28.556915 10775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:28.577991 10775 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:09:28.578913 10775 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:28.597069 10775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8eb286130fd64c8a85965ef420fa8834: Generated new TSK 0
I20260807 05:09:28.597833 10775 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
I20260807 05:09:28.634173 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:45143
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:40355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:28.743731 10787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:28.743999 10787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:28.744040 10787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:28.748173 10787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:28.748250 10787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:28.748318 10787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:09:28.753834 10787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:40355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:45143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:28.754916 10787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:28.755946 10787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:28.765437 10795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:28.765471 10793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:28.765458 10792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:28.765789 10787 server_base.cc:1034] running on GCE node
I20260807 05:09:28.766310 10787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:28.766944 10787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:28.768116 10787 hybrid_clock.cc:648] HybridClock initialized: now 1786079368768084 us; error 46 us; skew 500 ppm
Aug 07 05:09:28 dist-test-slave-49h3 krb5kdc[10678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079368, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:28.770905 10787 init.cc:377] Logged in from keytab as kudu/127.8.16.65@KRBTEST.COM (short username kudu)
I20260807 05:09:28.771792 10787 webserver.cc:460] Webserver started at http://127.8.16.65:44733/ using document root <none> and password file <none>
I20260807 05:09:28.772329 10787 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:28.772395 10787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:28.772589 10787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:28.774631 10787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "5fde96c56afc4e4b95dcae2579dcbfef"
format_stamp: "Formatted at 2026-08-07 05:09:28 on dist-test-slave-49h3"
server_key: "047f8664adcd6ba877683f648db60063"
server_key_iv: "2d87070f03b00c55c7d19ad848e80335"
server_key_version: "encryptionkey@0"
I20260807 05:09:28.775188 10787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "5fde96c56afc4e4b95dcae2579dcbfef"
format_stamp: "Formatted at 2026-08-07 05:09:28 on dist-test-slave-49h3"
server_key: "047f8664adcd6ba877683f648db60063"
server_key_iv: "2d87070f03b00c55c7d19ad848e80335"
server_key_version: "encryptionkey@0"
I20260807 05:09:28.778748 10787 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:09:28.781023 10802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:28.782207 10787 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:28.782316 10787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "5fde96c56afc4e4b95dcae2579dcbfef"
format_stamp: "Formatted at 2026-08-07 05:09:28 on dist-test-slave-49h3"
server_key: "047f8664adcd6ba877683f648db60063"
server_key_iv: "2d87070f03b00c55c7d19ad848e80335"
server_key_version: "encryptionkey@0"
I20260807 05:09:28.782406 10787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:28.799247 10787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:28.802335 10787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:28.802515 10787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:28.803084 10787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:28.803876 10787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:28.803925 10787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:28.803978 10787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:28.804006 10787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:28.812408 10787 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:42911
I20260807 05:09:28.812436 10919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:42911 every 8 connection(s)
I20260807 05:09:28.813252 10787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:09:28.820760 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 10787
I20260807 05:09:28.820883 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:09:28.821190 8257 external_mini_cluster.cc:1428] Setting key 2e55ac4e87e741825d42154ea79c2a49
Aug 07 05:09:28 dist-test-slave-49h3 krb5kdc[10678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079368, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:28.826264 10764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:28.814928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:50139 (local address 127.8.16.126:45143)
0807 05:09:28.815085 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:28.815090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:28.815865 (+ 775us) server_negotiation.cc:407] Connection header received
0807 05:09:28.817029 (+ 1164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:28.817032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:28.817079 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:28.817172 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:28.818232 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:28.818757 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:28.819887 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:28.820122 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:28.820219 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:28.822814 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:28.822817 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:28.822819 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:28.822841 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:28.824156 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:28.824770 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:28.824774 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:28.824775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:28.824854 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:28.825177 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:28.825181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:28.825182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:28.825358 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:28.825465 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:28.825949 (+ 484us) server_negotiation.cc:299] Negotiation successful
0807 05:09:28.826100 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 05:09:28.827083 10922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:28.815303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:45143 (local address 127.8.16.65:50139)
0807 05:09:28.815720 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:28.815761 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:28.816853 (+ 1092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:28.817300 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:28.817306 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:28.817655 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:28.818109 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:28.818122 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:28.818844 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:28.818846 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:28.819755 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:28.819764 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:28.820256 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:28.820258 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:28.820405 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:28.820994 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:28.821014 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:28.822661 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:28.824307 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:28.824313 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:28.824328 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:28.824667 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:28.824938 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:28.824941 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:28.824943 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:28.825078 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:28.825484 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:28.825491 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:28.825727 (+ 236us) client_negotiation.cc:769] Sending connection context
0807 05:09:28.825890 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 05:09:28.826233 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":110,"threads_started":1}
I20260807 05:09:28.828223 10920 heartbeater.cc:347] Connected to a master server at 127.8.16.126:45143
I20260807 05:09:28.828438 10920 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:28.828899 10920 heartbeater.cc:511] Master 127.8.16.126:45143 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.16.66@KRBTEST.COM; defaulting to no policy
I20260807 05:09:28.830509 10724 ts_manager.cc:194] Registered new tserver with Master: 5fde96c56afc4e4b95dcae2579dcbfef (127.8.16.65:42911)
I20260807 05:09:28.831676 10724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:50139
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
I20260807 05:09:28.846732 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:45143
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:40355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:28.954878 10925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:28.955150 10925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:28.955192 10925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:28.959246 10925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:28.959317 10925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:28.959389 10925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:09:28.964617 10925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:40355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:45143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:28.965737 10925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:28.966765 10925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:28.975901 10931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:28.975912 10930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:28.975901 10933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:28.976119 10925 server_base.cc:1034] running on GCE node
I20260807 05:09:28.976666 10925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:28.977308 10925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:28.978463 10925 hybrid_clock.cc:648] HybridClock initialized: now 1786079368978442 us; error 36 us; skew 500 ppm
Aug 07 05:09:28 dist-test-slave-49h3 krb5kdc[10678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079368, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:28.981030 10925 init.cc:377] Logged in from keytab as kudu/127.8.16.66@KRBTEST.COM (short username kudu)
I20260807 05:09:28.981861 10925 webserver.cc:460] Webserver started at http://127.8.16.66:44237/ using document root <none> and password file <none>
I20260807 05:09:28.982372 10925 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:28.982430 10925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:28.982589 10925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:28.984512 10925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "4d9beca6710d46e48b4ab707c13c513d"
format_stamp: "Formatted at 2026-08-07 05:09:28 on dist-test-slave-49h3"
server_key: "ebd5b6721b4a4d5fe0118a801b0815f1"
server_key_iv: "1a319f24882f21d3fc7c313d7aece782"
server_key_version: "encryptionkey@0"
I20260807 05:09:28.985134 10925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "4d9beca6710d46e48b4ab707c13c513d"
format_stamp: "Formatted at 2026-08-07 05:09:28 on dist-test-slave-49h3"
server_key: "ebd5b6721b4a4d5fe0118a801b0815f1"
server_key_iv: "1a319f24882f21d3fc7c313d7aece782"
server_key_version: "encryptionkey@0"
I20260807 05:09:28.988672 10925 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:09:28.990890 10940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:28.991973 10925 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:28.992062 10925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "4d9beca6710d46e48b4ab707c13c513d"
format_stamp: "Formatted at 2026-08-07 05:09:28 on dist-test-slave-49h3"
server_key: "ebd5b6721b4a4d5fe0118a801b0815f1"
server_key_iv: "1a319f24882f21d3fc7c313d7aece782"
server_key_version: "encryptionkey@0"
I20260807 05:09:28.992142 10925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:29.009724 10925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:29.012247 10925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:29.012391 10925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:29.012957 10925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:29.013723 10925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:29.013777 10925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:29.013821 10925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:29.013840 10925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:29.021935 10925 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:44397
I20260807 05:09:29.021963 11054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:44397 every 8 connection(s)
I20260807 05:09:29.022789 10925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
Aug 07 05:09:29 dist-test-slave-49h3 krb5kdc[10678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079368, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:29.032963 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 10925
I20260807 05:09:29.033088 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:09:29.033421 8257 external_mini_cluster.cc:1428] Setting key c1ff9c5831606775ca3ba0aa31223fdb
I20260807 05:09:29.035558 10764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:29.024432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:56943 (local address 127.8.16.126:45143)
0807 05:09:29.024558 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:29.024562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:29.025317 (+ 755us) server_negotiation.cc:407] Connection header received
0807 05:09:29.026562 (+ 1245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:29.026565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:29.026621 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:29.026696 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:29.027793 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:29.028317 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:29.029402 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:29.029659 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:29.029746 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:29.032097 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:29.032100 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:29.032102 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:29.032127 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:29.033574 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:29.034035 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:29.034038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:29.034039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:29.034132 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:29.034499 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:29.034501 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:29.034502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:29.034690 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:29.034786 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:29.035302 (+ 516us) server_negotiation.cc:299] Negotiation successful
0807 05:09:29.035415 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 05:09:29.036334 11057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:29.024769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:45143 (local address 127.8.16.66:56943)
0807 05:09:29.025180 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:29.025220 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:29.026368 (+ 1148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:29.026874 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:29.026880 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:29.027210 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:29.027659 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:29.027671 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:29.028429 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:29.028432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:29.029289 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:29.029298 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:29.029814 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:29.029819 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:29.029975 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:29.030501 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:29.030517 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:29.031938 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:29.033677 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:29.033682 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:29.033694 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:29.033954 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:29.034231 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:29.034234 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:29.034236 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:29.034412 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:29.034790 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:29.034797 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:29.035065 (+ 268us) client_negotiation.cc:769] Sending connection context
0807 05:09:29.035228 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 05:09:29.035474 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":100,"threads_started":1}
I20260807 05:09:29.037660 11055 heartbeater.cc:347] Connected to a master server at 127.8.16.126:45143
I20260807 05:09:29.037909 11055 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:29.038434 11055 heartbeater.cc:511] Master 127.8.16.126:45143 requested a full tablet report, sending...
I20260807 05:09:29.039484 10724 ts_manager.cc:194] Registered new tserver with Master: 4d9beca6710d46e48b4ab707c13c513d (127.8.16.66:44397)
I20260807 05:09:29.040238 10724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:56943
WARNING: no policy specified for kudu/127.8.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
I20260807 05:09:29.060403 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:45143
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:40355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:29.168998 11061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:29.169252 11061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:29.169298 11061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:29.173216 11061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:29.173269 11061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:29.173337 11061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:09:29.178525 11061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:40355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:45143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:29.179554 11061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:29.180528 11061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:29.189782 11069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:29.189918 11061 server_base.cc:1034] running on GCE node
W20260807 05:09:29.189774 11066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:29.189774 11067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:29.190460 11061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:29.191067 11061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:29.192214 11061 hybrid_clock.cc:648] HybridClock initialized: now 1786079369192190 us; error 38 us; skew 500 ppm
Aug 07 05:09:29 dist-test-slave-49h3 krb5kdc[10678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079369, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:29.194831 11061 init.cc:377] Logged in from keytab as kudu/127.8.16.67@KRBTEST.COM (short username kudu)
I20260807 05:09:29.195729 11061 webserver.cc:460] Webserver started at http://127.8.16.67:35825/ using document root <none> and password file <none>
I20260807 05:09:29.196266 11061 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:29.196326 11061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:29.196488 11061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:29.198525 11061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "e6c4fb6e1aa04c86ab697ec2c7c56ccd"
format_stamp: "Formatted at 2026-08-07 05:09:29 on dist-test-slave-49h3"
server_key: "4c8c9c4ba332bbd1bdce02a0b188ac3b"
server_key_iv: "8029905a23b0ccbea24bb5c1a09e5bfa"
server_key_version: "encryptionkey@0"
I20260807 05:09:29.199083 11061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "e6c4fb6e1aa04c86ab697ec2c7c56ccd"
format_stamp: "Formatted at 2026-08-07 05:09:29 on dist-test-slave-49h3"
server_key: "4c8c9c4ba332bbd1bdce02a0b188ac3b"
server_key_iv: "8029905a23b0ccbea24bb5c1a09e5bfa"
server_key_version: "encryptionkey@0"
I20260807 05:09:29.202682 11061 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:09:29.204982 11076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:29.206893 11061 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:09:29.207044 11061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "e6c4fb6e1aa04c86ab697ec2c7c56ccd"
format_stamp: "Formatted at 2026-08-07 05:09:29 on dist-test-slave-49h3"
server_key: "4c8c9c4ba332bbd1bdce02a0b188ac3b"
server_key_iv: "8029905a23b0ccbea24bb5c1a09e5bfa"
server_key_version: "encryptionkey@0"
I20260807 05:09:29.207290 11061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:29.224457 11061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:29.227787 11061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:29.227980 11061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:29.228559 11061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:29.229580 11061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:29.229637 11061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:29.229691 11061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:29.229712 11061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:29.240010 11061 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:46117
I20260807 05:09:29.240058 11192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:46117 every 8 connection(s)
I20260807 05:09:29.240947 11061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:09:29.246829 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 11061
I20260807 05:09:29.246950 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:09:29.247242 8257 external_mini_cluster.cc:1428] Setting key 66a6b661891891fb97e4288a9ba28611
Aug 07 05:09:29 dist-test-slave-49h3 krb5kdc[10678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079369, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:29.258107 10764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:29.243156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:44963 (local address 127.8.16.126:45143)
0807 05:09:29.243304 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:29.243309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:29.244475 (+ 1166us) server_negotiation.cc:407] Connection header received
0807 05:09:29.245702 (+ 1227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:29.245705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:29.245758 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:29.245848 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:29.247441 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:29.248267 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:29.250610 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:29.250954 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:29.251028 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:29.253972 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:29.253976 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:29.253978 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:29.254006 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:29.255553 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:29.256186 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:29.256190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:29.256192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:29.256286 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:29.256709 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:29.256712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:29.256714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:29.256951 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:29.257035 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:29.257786 (+ 751us) server_negotiation.cc:299] Negotiation successful
0807 05:09:29.257930 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 05:09:29.259841 11195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:29.243571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:45143 (local address 127.8.16.67:44963)
0807 05:09:29.244300 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:29.244359 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:29.245409 (+ 1050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:29.246033 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:29.246043 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:29.246556 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:29.247280 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:29.247300 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:29.248574 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:29.248577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:29.250478 (+ 1901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:29.250489 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:29.251218 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:29.251221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:29.251408 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:29.251976 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:29.251995 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:29.253794 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:29.255694 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:29.255699 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:29.255715 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:29.256060 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:29.256398 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:29.256401 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:29.256403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:29.256580 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:29.257108 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:29.257115 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:29.257442 (+ 327us) client_negotiation.cc:769] Sending connection context
0807 05:09:29.258351 (+ 909us) client_negotiation.cc:240] Negotiation successful
0807 05:09:29.258730 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":102,"threads_started":1}
I20260807 05:09:29.261157 11193 heartbeater.cc:347] Connected to a master server at 127.8.16.126:45143
I20260807 05:09:29.261428 11193 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:29.262001 11193 heartbeater.cc:511] Master 127.8.16.126:45143 requested a full tablet report, sending...
I20260807 05:09:29.262951 10724 ts_manager.cc:194] Registered new tserver with Master: e6c4fb6e1aa04c86ab697ec2c7c56ccd (127.8.16.67:46117)
I20260807 05:09:29.263489 10724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:44963
I20260807 05:09:29.273224 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 05:09:29.286556 10764 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:29.285158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46816 (local address 127.8.16.126:45143)
0807 05:09:29.285302 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:29.285306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:29.285470 (+ 164us) server_negotiation.cc:407] Connection header received
0807 05:09:29.285608 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:29.285610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:29.285663 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:29.285774 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:29.286450 (+ 676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46816: BlockingRecv error: recv got EOF from 127.0.0.1:46816 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 05:09:29.287633 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 10787
I20260807 05:09:29.293429 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 10925
I20260807 05:09:29.299554 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 11061
I20260807 05:09:29.306517 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 10691
2026-08-07T05:09:29Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2051 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:09:29 dist-test-slave-49h3 krb5kdc[11205](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:09:29 dist-test-slave-49h3 krb5kdc[11205](info): set up 2 sockets
Aug 07 05:09:29 dist-test-slave-49h3 krb5kdc[11205](info): commencing operation
krb5kdc: starting...
W20260807 05:09:30.289502 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.962s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:09:30 dist-test-slave-49h3 krb5kdc[11205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:09:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:09:30.406749 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:43977
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:39767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:43977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:30.513438 11218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:30.513720 11218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:30.513782 11218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:30.517779 11218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:30.517835 11218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:30.517863 11218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:30.517882 11218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:30.517904 11218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:30.523044 11218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:39767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:43977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:43977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:30.524075 11218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:30.525208 11218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:30.533198 11223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:30.533208 11227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:30.533209 11224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:30.533408 11218 server_base.cc:1034] running on GCE node
I20260807 05:09:30.533983 11218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:30.535032 11218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:30.536185 11218 hybrid_clock.cc:648] HybridClock initialized: now 1786079370536174 us; error 25 us; skew 500 ppm
Aug 07 05:09:30 dist-test-slave-49h3 krb5kdc[11205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:30.542615 11218 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:09:30.543478 11218 webserver.cc:460] Webserver started at http://127.8.16.126:46805/ using document root <none> and password file <none>
I20260807 05:09:30.543998 11218 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:30.544060 11218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:30.544239 11218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:30.546285 11218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "2101f8de4b7e4ff49591f05db75d2646"
format_stamp: "Formatted at 2026-08-07 05:09:30 on dist-test-slave-49h3"
server_key: "f62ca3e578a321d5993c338b1a091f1f"
server_key_iv: "c35d3e54fbbcce6f30e8709d69a01b84"
server_key_version: "encryptionkey@0"
I20260807 05:09:30.546795 11218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "2101f8de4b7e4ff49591f05db75d2646"
format_stamp: "Formatted at 2026-08-07 05:09:30 on dist-test-slave-49h3"
server_key: "f62ca3e578a321d5993c338b1a091f1f"
server_key_iv: "c35d3e54fbbcce6f30e8709d69a01b84"
server_key_version: "encryptionkey@0"
I20260807 05:09:30.550251 11218 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:09:30.552440 11234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:30.553589 11218 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:30.553694 11218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "2101f8de4b7e4ff49591f05db75d2646"
format_stamp: "Formatted at 2026-08-07 05:09:30 on dist-test-slave-49h3"
server_key: "f62ca3e578a321d5993c338b1a091f1f"
server_key_iv: "c35d3e54fbbcce6f30e8709d69a01b84"
server_key_version: "encryptionkey@0"
I20260807 05:09:30.553785 11218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:30.568552 11218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:30.571473 11218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:30.571655 11218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:30.578316 11218 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:43977
I20260807 05:09:30.578562 11286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:43977 every 8 connection(s)
I20260807 05:09:30.579231 11218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:30.581833 11287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:30.582733 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 11218
I20260807 05:09:30.582837 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:09:30.583083 8257 external_mini_cluster.cc:1428] Setting key dc0689cf52890bffb31619a130233535
I20260807 05:09:30.587731 11287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646: Bootstrap starting.
I20260807 05:09:30.590267 11287 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:30.591150 11287 log.cc:824] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646: Log is configured to *not* fsync() on all Append() calls
Aug 07 05:09:30 dist-test-slave-49h3 krb5kdc[11205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:30.593247 11287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646: No bootstrap required, opened a new log
I20260807 05:09:30.596716 11290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:30.584276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51768 (local address 127.8.16.126:43977)
0807 05:09:30.584673 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:30.584688 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:30.584722 (+ 34us) server_negotiation.cc:407] Connection header received
0807 05:09:30.585331 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:30.585347 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:30.585707 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:30.585981 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:30.586929 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:30.587910 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:30.589045 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:30.589592 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:30.589723 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:30.592438 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:30.592444 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:30.592456 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:30.592482 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:30.594083 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:30.594498 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:30.594504 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:30.594510 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:30.594616 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:30.594874 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:30.594877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:30.594879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:30.595255 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:30.595429 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:30.595710 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 05:09:30.596003 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":110,"threads_started":1}
I20260807 05:09:30.596719 11287 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2101f8de4b7e4ff49591f05db75d2646" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43977 } }
I20260807 05:09:30.596975 11287 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:30.597028 11287 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2101f8de4b7e4ff49591f05db75d2646, State: Initialized, Role: FOLLOWER
I20260807 05:09:30.597437 11287 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2101f8de4b7e4ff49591f05db75d2646" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43977 } }
I20260807 05:09:30.597591 11287 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:09:30.597651 11287 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:09:30.597744 11287 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:30.598870 11287 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2101f8de4b7e4ff49591f05db75d2646" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43977 } }
I20260807 05:09:30.599202 11287 leader_election.cc:302] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2101f8de4b7e4ff49591f05db75d2646; no voters:
I20260807 05:09:30.599534 11287 leader_election.cc:288] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:09:30.599655 11292 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:30.599675 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:09:30.599869 11292 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646 [term 1 LEADER]: Becoming Leader. State: Replica: 2101f8de4b7e4ff49591f05db75d2646, State: Running, Role: LEADER
I20260807 05:09:30.600196 11292 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2101f8de4b7e4ff49591f05db75d2646" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43977 } }
I20260807 05:09:30.600852 11287 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:30.601732 11295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2101f8de4b7e4ff49591f05db75d2646. Latest consensus state: current_term: 1 leader_uuid: "2101f8de4b7e4ff49591f05db75d2646" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2101f8de4b7e4ff49591f05db75d2646" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43977 } } }
I20260807 05:09:30.601876 11295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:30.602232 11294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2101f8de4b7e4ff49591f05db75d2646" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2101f8de4b7e4ff49591f05db75d2646" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43977 } } }
I20260807 05:09:30.602324 11294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646 [sys.catalog]: This master's current role is: LEADER
W20260807 05:09:30.605683 11309 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:09:30.605767 11309 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:09:30.605825 11303 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:30.606555 11303 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:30.612592 11303 catalog_manager.cc:1347] Generated new cluster ID: f87c83f4f91c4b2f91cfd94f96e1a3ea
I20260807 05:09:30.612672 11303 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:30.623457 11303 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:09:30.624718 11303 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:30.627197 11290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:30.616553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51772 (local address 127.8.16.126:43977)
0807 05:09:30.616665 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:30.616670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:30.621843 (+ 5173us) server_negotiation.cc:407] Connection header received
0807 05:09:30.621980 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:30.621984 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:30.622043 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:30.622141 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:30.622750 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:30.623553 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:30.624726 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:30.625098 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:30.625197 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:30.626190 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:30.626193 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:30.626197 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:09:30.626756 (+ 559us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:30.626792 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:09:30.626813 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:30.626902 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:30.626974 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 05:09:30.627057 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 05:09:30.638404 11303 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2101f8de4b7e4ff49591f05db75d2646: Generated new TSK 0
I20260807 05:09:30.639263 11303 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:09:30.645778 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 11218
2026-08-07T05:09:30Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1338 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:09:30 dist-test-slave-49h3 krb5kdc[11321](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:09:30 dist-test-slave-49h3 krb5kdc[11321](info): set up 2 sockets
Aug 07 05:09:30 dist-test-slave-49h3 krb5kdc[11321](info): commencing operation
krb5kdc: starting...
W20260807 05:09:31.629153 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.964s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:09:31 dist-test-slave-49h3 krb5kdc[11321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:09:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:09:31.748975 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:33703
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:44879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:33703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:31.855732 11334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:31.856067 11334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:31.856148 11334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:31.860260 11334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:31.860330 11334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:31.860349 11334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:31.860364 11334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:31.860378 11334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:31.865809 11334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:44879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:33703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:33703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:31.866941 11334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:31.867964 11334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:31.876340 11339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:31.876340 11342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:31.876438 11340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:31.876724 11334 server_base.cc:1034] running on GCE node
I20260807 05:09:31.877353 11334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:31.878532 11334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:31.879711 11334 hybrid_clock.cc:648] HybridClock initialized: now 1786079371879697 us; error 34 us; skew 500 ppm
Aug 07 05:09:31 dist-test-slave-49h3 krb5kdc[11321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079371, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:31.882628 11334 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:09:31.883565 11334 webserver.cc:460] Webserver started at http://127.8.16.126:43187/ using document root <none> and password file <none>
I20260807 05:09:31.884239 11334 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:31.884315 11334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:31.884495 11334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:31.886651 11334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "dbe9ff6adb064ec0ad866e2712201bbb"
format_stamp: "Formatted at 2026-08-07 05:09:31 on dist-test-slave-49h3"
server_key: "b172cf89eae2c4460693cc7726b50ad7"
server_key_iv: "909bcffe3fda823516df09dbf09229b4"
server_key_version: "encryptionkey@0"
I20260807 05:09:31.887212 11334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "dbe9ff6adb064ec0ad866e2712201bbb"
format_stamp: "Formatted at 2026-08-07 05:09:31 on dist-test-slave-49h3"
server_key: "b172cf89eae2c4460693cc7726b50ad7"
server_key_iv: "909bcffe3fda823516df09dbf09229b4"
server_key_version: "encryptionkey@0"
I20260807 05:09:31.890946 11334 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:09:31.893268 11349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:31.894555 11334 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:31.894659 11334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "dbe9ff6adb064ec0ad866e2712201bbb"
format_stamp: "Formatted at 2026-08-07 05:09:31 on dist-test-slave-49h3"
server_key: "b172cf89eae2c4460693cc7726b50ad7"
server_key_iv: "909bcffe3fda823516df09dbf09229b4"
server_key_version: "encryptionkey@0"
I20260807 05:09:31.894764 11334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:31.909296 11334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:31.911999 11334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:31.912175 11334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:31.918819 11334 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:33703
I20260807 05:09:31.919070 11403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:33703 every 8 connection(s)
I20260807 05:09:31.919754 11334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:31.922415 11404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:31.925414 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 11334
I20260807 05:09:31.925550 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:09:31.925853 8257 external_mini_cluster.cc:1428] Setting key 9b58e5a3c0c8ee6c2cb9e65d0c9f20fd
I20260807 05:09:31.928158 11404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb: Bootstrap starting.
I20260807 05:09:31.930615 11404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:31.931470 11404 log.cc:824] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:31.933724 11404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb: No bootstrap required, opened a new log
Aug 07 05:09:31 dist-test-slave-49h3 krb5kdc[11321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:31.937731 11404 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe9ff6adb064ec0ad866e2712201bbb" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 33703 } }
I20260807 05:09:31.938017 11404 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:31.938102 11404 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe9ff6adb064ec0ad866e2712201bbb, State: Initialized, Role: FOLLOWER
I20260807 05:09:31.938597 11404 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe9ff6adb064ec0ad866e2712201bbb" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 33703 } }
I20260807 05:09:31.938810 11404 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:09:31.938885 11404 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:09:31.938979 11404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:31.940171 11404 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe9ff6adb064ec0ad866e2712201bbb" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 33703 } }
I20260807 05:09:31.940548 11404 leader_election.cc:302] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbe9ff6adb064ec0ad866e2712201bbb; no voters:
I20260807 05:09:31.940945 11404 leader_election.cc:288] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:09:31.941074 11409 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:31.941268 11409 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb [term 1 LEADER]: Becoming Leader. State: Replica: dbe9ff6adb064ec0ad866e2712201bbb, State: Running, Role: LEADER
I20260807 05:09:31.941587 11409 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe9ff6adb064ec0ad866e2712201bbb" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 33703 } }
I20260807 05:09:31.942296 11404 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:31.942595 11407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:31.927213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52850 (local address 127.8.16.126:33703)
0807 05:09:31.927595 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:31.927610 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:31.927645 (+ 35us) server_negotiation.cc:407] Connection header received
0807 05:09:31.928259 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:31.928282 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:31.928609 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:31.928876 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:31.929822 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:31.930847 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:31.931959 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:31.932533 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:31.932650 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:31.935445 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:31.935451 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:31.935470 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:31.935502 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:31.938513 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:31.939147 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:31.939153 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:31.939160 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:31.939320 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:31.939638 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:31.939642 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:31.939644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:31.940195 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:31.940405 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:31.941207 (+ 802us) server_negotiation.cc:299] Negotiation successful
0807 05:09:31.941575 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":103,"threads_started":1}
I20260807 05:09:31.942803 11409 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbe9ff6adb064ec0ad866e2712201bbb. Latest consensus state: current_term: 1 leader_uuid: "dbe9ff6adb064ec0ad866e2712201bbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe9ff6adb064ec0ad866e2712201bbb" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 33703 } } }
I20260807 05:09:31.942902 11409 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:31.943631 11411 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbe9ff6adb064ec0ad866e2712201bbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe9ff6adb064ec0ad866e2712201bbb" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 33703 } } }
I20260807 05:09:31.943859 11411 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb [sys.catalog]: This master's current role is: LEADER
W20260807 05:09:31.947131 11424 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:09:31.947198 11424 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:09:31.947259 11414 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:31.947901 11414 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:31.952926 11414 catalog_manager.cc:1347] Generated new cluster ID: f21f6bb68dc247af8241eab7223314c6
I20260807 05:09:31.952993 11414 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:31.963218 11414 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:09:31.964526 11414 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:31.976500 11414 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbe9ff6adb064ec0ad866e2712201bbb: Generated new TSK 0
I20260807 05:09:31.977415 11414 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
I20260807 05:09:32.008752 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:33703
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:44879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:32.119333 11430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:32.119618 11430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:32.119678 11430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:32.123665 11430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:32.123734 11430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:32.123816 11430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:09:32.129140 11430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:44879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:33703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:32.130249 11430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:32.131256 11430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:32.140406 11438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:32.140412 11435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:32.140398 11436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:32.140679 11430 server_base.cc:1034] running on GCE node
I20260807 05:09:32.141089 11430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:32.141722 11430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:32.142882 11430 hybrid_clock.cc:648] HybridClock initialized: now 1786079372142862 us; error 28 us; skew 500 ppm
Aug 07 05:09:32 dist-test-slave-49h3 krb5kdc[11321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079372, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:32.145709 11430 init.cc:377] Logged in from keytab as kudu/127.8.16.65@KRBTEST.COM (short username kudu)
I20260807 05:09:32.146553 11430 webserver.cc:460] Webserver started at http://127.8.16.65:37291/ using document root <none> and password file <none>
I20260807 05:09:32.147086 11430 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:32.147152 11430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:32.147343 11430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:32.149331 11430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "f5eea14502ea4a05bcdf3c00bf3a18d5"
format_stamp: "Formatted at 2026-08-07 05:09:32 on dist-test-slave-49h3"
server_key: "3caadc593d4ca5453832074af93b6f89"
server_key_iv: "9ca36a2f488d6d586bd82ac28c328e72"
server_key_version: "encryptionkey@0"
I20260807 05:09:32.149916 11430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "f5eea14502ea4a05bcdf3c00bf3a18d5"
format_stamp: "Formatted at 2026-08-07 05:09:32 on dist-test-slave-49h3"
server_key: "3caadc593d4ca5453832074af93b6f89"
server_key_iv: "9ca36a2f488d6d586bd82ac28c328e72"
server_key_version: "encryptionkey@0"
I20260807 05:09:32.153367 11430 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:09:32.155645 11445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:32.156860 11430 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:32.156970 11430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "f5eea14502ea4a05bcdf3c00bf3a18d5"
format_stamp: "Formatted at 2026-08-07 05:09:32 on dist-test-slave-49h3"
server_key: "3caadc593d4ca5453832074af93b6f89"
server_key_iv: "9ca36a2f488d6d586bd82ac28c328e72"
server_key_version: "encryptionkey@0"
I20260807 05:09:32.157058 11430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:32.185289 11430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:32.188493 11430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:32.188686 11430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:32.189244 11430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:32.190049 11430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:32.190099 11430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:32.190136 11430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:32.190156 11430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:32.198266 11430 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:33687
I20260807 05:09:32.198309 11563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:33687 every 8 connection(s)
I20260807 05:09:32.199173 11430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:09:32.205892 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 11430
I20260807 05:09:32.206003 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:09:32.206329 8257 external_mini_cluster.cc:1428] Setting key 1680f67317668f6f12182d60d31145a3
Aug 07 05:09:32 dist-test-slave-49h3 krb5kdc[11321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079372, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:32.212387 11407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:32.200849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:53571 (local address 127.8.16.126:33703)
0807 05:09:32.200969 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:32.200972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:32.201800 (+ 828us) server_negotiation.cc:407] Connection header received
0807 05:09:32.203121 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:32.203125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:32.203183 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:32.203260 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:32.204379 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:32.204910 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:32.206048 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:32.206396 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:32.206512 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:32.209137 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:32.209141 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:32.209143 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:32.209166 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:32.210525 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:32.210987 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:32.210991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:32.210992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:32.211070 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:32.211377 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:32.211381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:32.211382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:32.211553 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:32.211645 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:32.212097 (+ 452us) server_negotiation.cc:299] Negotiation successful
0807 05:09:32.212221 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 05:09:32.213181 11566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:32.201187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:33703 (local address 127.8.16.65:53571)
0807 05:09:32.201651 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:32.201692 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:32.202917 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:32.203426 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:32.203432 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:32.203787 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:32.204235 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:32.204248 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:32.205029 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:32.205031 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:32.205918 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:32.205928 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:32.206540 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:32.206543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:32.206685 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:32.207291 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:32.207310 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:32.208982 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:32.210640 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:32.210645 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:32.210659 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:32.210896 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:32.211153 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:32.211156 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:32.211158 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:32.211292 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:32.211647 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:32.211653 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:32.211880 (+ 227us) client_negotiation.cc:769] Sending connection context
0807 05:09:32.212039 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 05:09:32.212349 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":114,"threads_started":1}
I20260807 05:09:32.214483 11564 heartbeater.cc:347] Connected to a master server at 127.8.16.126:33703
I20260807 05:09:32.214690 11564 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:32.215250 11564 heartbeater.cc:511] Master 127.8.16.126:33703 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.16.66@KRBTEST.COM; defaulting to no policy
I20260807 05:09:32.216732 11367 ts_manager.cc:194] Registered new tserver with Master: f5eea14502ea4a05bcdf3c00bf3a18d5 (127.8.16.65:33687)
I20260807 05:09:32.217854 11367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:53571
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
I20260807 05:09:32.232192 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:33703
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:44879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:32.342021 11569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:32.342300 11569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:32.342363 11569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:32.346479 11569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:32.346541 11569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:32.346617 11569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:09:32.352126 11569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:44879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:33703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:32.353195 11569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:32.354254 11569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:32.363546 11575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:32.363541 11577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:32.363758 11569 server_base.cc:1034] running on GCE node
W20260807 05:09:32.363546 11574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:32.364274 11569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:32.364924 11569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:32.366122 11569 hybrid_clock.cc:648] HybridClock initialized: now 1786079372366100 us; error 31 us; skew 500 ppm
Aug 07 05:09:32 dist-test-slave-49h3 krb5kdc[11321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079372, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:32.368695 11569 init.cc:377] Logged in from keytab as kudu/127.8.16.66@KRBTEST.COM (short username kudu)
I20260807 05:09:32.369549 11569 webserver.cc:460] Webserver started at http://127.8.16.66:36339/ using document root <none> and password file <none>
I20260807 05:09:32.370052 11569 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:32.370119 11569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:32.370285 11569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:32.372241 11569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "930c99ba324641eb92e7d987c3dffbb3"
format_stamp: "Formatted at 2026-08-07 05:09:32 on dist-test-slave-49h3"
server_key: "9fdedc21f9ac1560c7e875877eb8771c"
server_key_iv: "510c2fdef269ef1d7246616da803d772"
server_key_version: "encryptionkey@0"
I20260807 05:09:32.372745 11569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "930c99ba324641eb92e7d987c3dffbb3"
format_stamp: "Formatted at 2026-08-07 05:09:32 on dist-test-slave-49h3"
server_key: "9fdedc21f9ac1560c7e875877eb8771c"
server_key_iv: "510c2fdef269ef1d7246616da803d772"
server_key_version: "encryptionkey@0"
I20260807 05:09:32.376293 11569 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:09:32.378556 11584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:32.379729 11569 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:32.379829 11569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "930c99ba324641eb92e7d987c3dffbb3"
format_stamp: "Formatted at 2026-08-07 05:09:32 on dist-test-slave-49h3"
server_key: "9fdedc21f9ac1560c7e875877eb8771c"
server_key_iv: "510c2fdef269ef1d7246616da803d772"
server_key_version: "encryptionkey@0"
I20260807 05:09:32.379932 11569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:32.412382 11569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:32.415482 11569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:32.415675 11569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:32.416316 11569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:32.417155 11569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:32.417202 11569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:32.417245 11569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:32.417274 11569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:32.425724 11569 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:42241
I20260807 05:09:32.425752 11699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:42241 every 8 connection(s)
I20260807 05:09:32.426620 11569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:09:32.429039 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 11569
I20260807 05:09:32.429176 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:09:32.429526 8257 external_mini_cluster.cc:1428] Setting key b5f4f60bd3863f4aedc25fad54925d36
Aug 07 05:09:32 dist-test-slave-49h3 krb5kdc[11321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079372, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.16.67@KRBTEST.COM; defaulting to no policy
I20260807 05:09:32.440944 11407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:32.428311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:51101 (local address 127.8.16.126:33703)
0807 05:09:32.428428 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:32.428432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:32.429272 (+ 840us) server_negotiation.cc:407] Connection header received
0807 05:09:32.430782 (+ 1510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:32.430784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:32.430824 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:32.430890 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:32.432064 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:32.432803 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:32.434453 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:32.434678 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:32.434764 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:32.437410 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:32.437414 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:32.437417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:32.437475 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:32.438955 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:32.439427 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:32.439432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:32.439434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:32.439522 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:32.439820 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:32.439824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:32.439826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:32.440066 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:32.440177 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:32.440686 (+ 509us) server_negotiation.cc:299] Negotiation successful
0807 05:09:32.440817 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 05:09:32.441897 11703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:32.428629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:33703 (local address 127.8.16.66:51101)
0807 05:09:32.429104 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:32.429162 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:32.430578 (+ 1416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:32.431091 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:32.431099 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:32.431454 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:32.431919 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:32.431932 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:32.432956 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:32.432960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:32.434340 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:32.434351 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:32.434825 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:32.434829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:32.435016 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:32.435603 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:32.435624 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:32.437235 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:32.439066 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:32.439071 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:32.439083 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:32.439330 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:32.439606 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:32.439608 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:32.439609 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:32.439737 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:32.440171 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:32.440176 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:32.440417 (+ 241us) client_negotiation.cc:769] Sending connection context
0807 05:09:32.440571 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 05:09:32.440891 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":135,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.67@KRBTEST.COM" created.
I20260807 05:09:32.443106 11700 heartbeater.cc:347] Connected to a master server at 127.8.16.126:33703
I20260807 05:09:32.443449 11700 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:32.444022 11700 heartbeater.cc:511] Master 127.8.16.126:33703 requested a full tablet report, sending...
I20260807 05:09:32.444983 11367 ts_manager.cc:194] Registered new tserver with Master: 930c99ba324641eb92e7d987c3dffbb3 (127.8.16.66:42241)
I20260807 05:09:32.445559 11367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:51101
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
I20260807 05:09:32.456583 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:33703
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:44879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:32.564546 11706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:32.564827 11706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:32.564888 11706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:32.568895 11706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:32.568956 11706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:32.569034 11706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:09:32.574246 11706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:44879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:33703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:32.575270 11706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:32.576268 11706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:32.585510 11711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:32.585484 11712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:32.585546 11714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:32.585628 11706 server_base.cc:1034] running on GCE node
I20260807 05:09:32.586114 11706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:32.586722 11706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:32.587899 11706 hybrid_clock.cc:648] HybridClock initialized: now 1786079372587880 us; error 28 us; skew 500 ppm
Aug 07 05:09:32 dist-test-slave-49h3 krb5kdc[11321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079372, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:32.590588 11706 init.cc:377] Logged in from keytab as kudu/127.8.16.67@KRBTEST.COM (short username kudu)
I20260807 05:09:32.591360 11706 webserver.cc:460] Webserver started at http://127.8.16.67:46525/ using document root <none> and password file <none>
I20260807 05:09:32.591852 11706 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:32.591908 11706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:32.592072 11706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:32.594061 11706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "c0db422224c64637ab233dc477c43c9b"
format_stamp: "Formatted at 2026-08-07 05:09:32 on dist-test-slave-49h3"
server_key: "e7693a5a3920106cc1ddd404ac009970"
server_key_iv: "b040ff390d55600afd9aa9bfe9c5fc9b"
server_key_version: "encryptionkey@0"
I20260807 05:09:32.594556 11706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "c0db422224c64637ab233dc477c43c9b"
format_stamp: "Formatted at 2026-08-07 05:09:32 on dist-test-slave-49h3"
server_key: "e7693a5a3920106cc1ddd404ac009970"
server_key_iv: "b040ff390d55600afd9aa9bfe9c5fc9b"
server_key_version: "encryptionkey@0"
I20260807 05:09:32.598028 11706 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:09:32.600401 11721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:32.601528 11706 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:32.601627 11706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "c0db422224c64637ab233dc477c43c9b"
format_stamp: "Formatted at 2026-08-07 05:09:32 on dist-test-slave-49h3"
server_key: "e7693a5a3920106cc1ddd404ac009970"
server_key_iv: "b040ff390d55600afd9aa9bfe9c5fc9b"
server_key_version: "encryptionkey@0"
I20260807 05:09:32.601709 11706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:32.619638 11706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:32.622920 11706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:32.623097 11706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:32.623633 11706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:32.624413 11706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:32.624459 11706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:32.624497 11706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:32.624518 11706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:32.632936 11706 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:42981
I20260807 05:09:32.632952 11838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:42981 every 8 connection(s)
I20260807 05:09:32.633872 11706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
Aug 07 05:09:32 dist-test-slave-49h3 krb5kdc[11321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079372, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:32.643294 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 11706
I20260807 05:09:32.643404 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:09:32.643746 8257 external_mini_cluster.cc:1428] Setting key cd431070130a3a46ebf7fe2e862ab35a
I20260807 05:09:32.647432 11407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:32.635634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:35781 (local address 127.8.16.126:33703)
0807 05:09:32.635760 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:32.635764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:32.636561 (+ 797us) server_negotiation.cc:407] Connection header received
0807 05:09:32.637963 (+ 1402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:32.637967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:32.638028 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:32.638239 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:32.639401 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:32.639927 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:32.641038 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:32.641264 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:32.641351 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:32.644026 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:32.644031 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:32.644034 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:32.644065 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:32.645661 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:32.646200 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:32.646203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:32.646204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:32.646286 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:32.646548 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:32.646550 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:32.646551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:32.646722 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:32.646792 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:32.647215 (+ 423us) server_negotiation.cc:299] Negotiation successful
0807 05:09:32.647316 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 05:09:32.648214 11841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:32.635941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:33703 (local address 127.8.16.67:35781)
0807 05:09:32.636406 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:32.636449 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:32.637750 (+ 1301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:32.638399 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:32.638405 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:32.638783 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:32.639255 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:32.639267 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:32.640046 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:32.640049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:32.640923 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:32.640932 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:32.641401 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:32.641405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:32.641612 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:32.642167 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:32.642190 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:32.643843 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:32.645761 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:32.645765 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:32.645778 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:32.646108 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:32.646356 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:32.646358 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:32.646360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:32.646487 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:32.646791 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:32.646796 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:32.647022 (+ 226us) client_negotiation.cc:769] Sending connection context
0807 05:09:32.647174 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 05:09:32.647437 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":118,"threads_started":1}
I20260807 05:09:32.649210 11839 heartbeater.cc:347] Connected to a master server at 127.8.16.126:33703
I20260807 05:09:32.649468 11839 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:32.649958 11839 heartbeater.cc:511] Master 127.8.16.126:33703 requested a full tablet report, sending...
I20260807 05:09:32.650889 11367 ts_manager.cc:194] Registered new tserver with Master: c0db422224c64637ab233dc477c43c9b (127.8.16.67:42981)
I20260807 05:09:32.651422 11367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:35781
I20260807 05:09:32.657582 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 05:09:32 dist-test-slave-49h3 krb5kdc[11321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079372, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 05:09:32 dist-test-slave-49h3 krb5kdc[11321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079372, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:32.682912 11407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:32.674697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52858 (local address 127.8.16.126:33703)
0807 05:09:32.674831 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:32.674836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:32.675051 (+ 215us) server_negotiation.cc:407] Connection header received
0807 05:09:32.675190 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:32.675194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:32.675239 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:32.675307 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:32.676031 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:32.676568 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:32.677427 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:32.677684 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:32.677780 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:32.680070 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:32.680073 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:32.680076 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:32.680102 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:32.681428 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:32.681876 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:32.681880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:32.681881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:32.681965 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:32.682171 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:32.682173 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:32.682175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:32.682392 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:32.682490 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:32.682657 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 05:09:32.682788 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 05:09:32.683282 11367 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52858
I20260807 05:09:32.684235 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 11430
I20260807 05:09:32.693636 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 11569
I20260807 05:09:32.699090 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 11706
I20260807 05:09:32.704396 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 11334
2026-08-07T05:09:32Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2059 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:09:32 dist-test-slave-49h3 krb5kdc[11852](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:09:32 dist-test-slave-49h3 krb5kdc[11852](info): set up 2 sockets
Aug 07 05:09:32 dist-test-slave-49h3 krb5kdc[11852](info): commencing operation
krb5kdc: starting...
W20260807 05:09:33.683300 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.958s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:09:33 dist-test-slave-49h3 krb5kdc[11852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:09:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:09:33.800079 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:42657
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:38291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:42657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:33.909389 11866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:33.909716 11866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:33.909757 11866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:33.913761 11866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:33.913825 11866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:33.913843 11866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:33.913858 11866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:33.913878 11866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:33.919245 11866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:38291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:42657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:42657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:33.920300 11866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:33.921298 11866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:33.929309 11874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:33.929339 11872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:33.929327 11871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:33.929510 11866 server_base.cc:1034] running on GCE node
I20260807 05:09:33.930132 11866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:33.931185 11866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:33.932343 11866 hybrid_clock.cc:648] HybridClock initialized: now 1786079373932330 us; error 25 us; skew 500 ppm
Aug 07 05:09:33 dist-test-slave-49h3 krb5kdc[11852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079373, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:33.935259 11866 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:09:33.936372 11866 webserver.cc:460] Webserver started at http://127.8.16.126:33209/ using document root <none> and password file <none>
I20260807 05:09:33.937073 11866 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:33.937161 11866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:33.937395 11866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:33.940143 11866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "a0bd0aa9f84c4581890fd0e4fcc8aca5"
format_stamp: "Formatted at 2026-08-07 05:09:33 on dist-test-slave-49h3"
server_key: "94d53d3e410b17771b98256bd3f8c1ff"
server_key_iv: "557796dce211964f023bf4ea53d1d757"
server_key_version: "encryptionkey@0"
I20260807 05:09:33.940834 11866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "a0bd0aa9f84c4581890fd0e4fcc8aca5"
format_stamp: "Formatted at 2026-08-07 05:09:33 on dist-test-slave-49h3"
server_key: "94d53d3e410b17771b98256bd3f8c1ff"
server_key_iv: "557796dce211964f023bf4ea53d1d757"
server_key_version: "encryptionkey@0"
I20260807 05:09:33.945433 11866 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 05:09:33.947844 11881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:33.949020 11866 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 05:09:33.949136 11866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "a0bd0aa9f84c4581890fd0e4fcc8aca5"
format_stamp: "Formatted at 2026-08-07 05:09:33 on dist-test-slave-49h3"
server_key: "94d53d3e410b17771b98256bd3f8c1ff"
server_key_iv: "557796dce211964f023bf4ea53d1d757"
server_key_version: "encryptionkey@0"
I20260807 05:09:33.949247 11866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:33.964818 11866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:33.967734 11866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:33.967926 11866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:33.974684 11866 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:42657
I20260807 05:09:33.974713 11934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:42657 every 8 connection(s)
I20260807 05:09:33.975662 11866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:33.976284 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 11866
I20260807 05:09:33.976393 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:09:33.976650 8257 external_mini_cluster.cc:1428] Setting key beff17146b213d5d31b20f41f9d2ebd5
I20260807 05:09:33.978808 11935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:33.984130 11935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0bd0aa9f84c4581890fd0e4fcc8aca5: Bootstrap starting.
Aug 07 05:09:33 dist-test-slave-49h3 krb5kdc[11852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:33.986380 11935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0bd0aa9f84c4581890fd0e4fcc8aca5: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:33.987074 11935 log.cc:824] T 00000000000000000000000000000000 P a0bd0aa9f84c4581890fd0e4fcc8aca5: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:33.988924 11935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0bd0aa9f84c4581890fd0e4fcc8aca5: No bootstrap required, opened a new log
I20260807 05:09:33.989658 11939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:33.978065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44840 (local address 127.8.16.126:42657)
0807 05:09:33.978504 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:33.978518 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:33.978553 (+ 35us) server_negotiation.cc:407] Connection header received
0807 05:09:33.979204 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:33.979221 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:33.979545 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:33.979841 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:33.980644 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:33.981668 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:33.982469 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:33.982912 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:33.983061 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:33.985360 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:33.985365 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:33.985377 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:33.985405 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:33.987016 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:33.987449 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:33.987454 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:33.987458 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:33.987562 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:33.987808 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:33.987811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:33.987813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:33.988191 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:33.988371 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:33.988629 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 05:09:33.988900 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":147,"threads_started":1}
I20260807 05:09:33.992818 11935 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0bd0aa9f84c4581890fd0e4fcc8aca5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0bd0aa9f84c4581890fd0e4fcc8aca5" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42657 } }
I20260807 05:09:33.993031 11935 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0bd0aa9f84c4581890fd0e4fcc8aca5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:33.993098 11935 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0bd0aa9f84c4581890fd0e4fcc8aca5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0bd0aa9f84c4581890fd0e4fcc8aca5, State: Initialized, Role: FOLLOWER
I20260807 05:09:33.993510 11935 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0bd0aa9f84c4581890fd0e4fcc8aca5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0bd0aa9f84c4581890fd0e4fcc8aca5" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42657 } }
I20260807 05:09:33.993639 11935 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0bd0aa9f84c4581890fd0e4fcc8aca5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:09:33.993708 11935 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0bd0aa9f84c4581890fd0e4fcc8aca5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:09:33.993803 11935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0bd0aa9f84c4581890fd0e4fcc8aca5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:33.994676 11935 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0bd0aa9f84c4581890fd0e4fcc8aca5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0bd0aa9f84c4581890fd0e4fcc8aca5" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42657 } }
I20260807 05:09:33.994979 11935 leader_election.cc:302] T 00000000000000000000000000000000 P a0bd0aa9f84c4581890fd0e4fcc8aca5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0bd0aa9f84c4581890fd0e4fcc8aca5; no voters:
I20260807 05:09:33.995328 11935 leader_election.cc:288] T 00000000000000000000000000000000 P a0bd0aa9f84c4581890fd0e4fcc8aca5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:09:33.995433 11941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0bd0aa9f84c4581890fd0e4fcc8aca5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:33.995651 11941 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0bd0aa9f84c4581890fd0e4fcc8aca5 [term 1 LEADER]: Becoming Leader. State: Replica: a0bd0aa9f84c4581890fd0e4fcc8aca5, State: Running, Role: LEADER
I20260807 05:09:33.995951 11941 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0bd0aa9f84c4581890fd0e4fcc8aca5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0bd0aa9f84c4581890fd0e4fcc8aca5" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42657 } }
I20260807 05:09:33.996447 11935 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0bd0aa9f84c4581890fd0e4fcc8aca5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:33.997134 11943 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0bd0aa9f84c4581890fd0e4fcc8aca5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0bd0aa9f84c4581890fd0e4fcc8aca5. Latest consensus state: current_term: 1 leader_uuid: "a0bd0aa9f84c4581890fd0e4fcc8aca5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0bd0aa9f84c4581890fd0e4fcc8aca5" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42657 } } }
I20260807 05:09:33.997255 11943 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0bd0aa9f84c4581890fd0e4fcc8aca5 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:33.997538 11942 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0bd0aa9f84c4581890fd0e4fcc8aca5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0bd0aa9f84c4581890fd0e4fcc8aca5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0bd0aa9f84c4581890fd0e4fcc8aca5" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42657 } } }
I20260807 05:09:33.997627 11942 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0bd0aa9f84c4581890fd0e4fcc8aca5 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:33.997692 11949 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:34.000608 11949 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:34.005960 11949 catalog_manager.cc:1347] Generated new cluster ID: 578f21d00128494cb5857c20366923da
I20260807 05:09:34.006026 11949 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:34.013501 11949 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:09:34.014413 11949 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:34.022262 11949 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0bd0aa9f84c4581890fd0e4fcc8aca5: Generated new TSK 0
I20260807 05:09:34.022902 11949 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
I20260807 05:09:34.056438 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:42657
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:38291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:34.187222 11962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:34.187484 11962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:34.187529 11962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:34.191535 11962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:34.191596 11962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:34.191669 11962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:09:34.196888 11962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:38291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:42657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:34.198055 11962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:34.199074 11962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:34.208560 11968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:34.208570 11970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:34.208570 11967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:34.208923 11962 server_base.cc:1034] running on GCE node
I20260807 05:09:34.209276 11962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:34.209903 11962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:34.211054 11962 hybrid_clock.cc:648] HybridClock initialized: now 1786079374211036 us; error 34 us; skew 500 ppm
Aug 07 05:09:34 dist-test-slave-49h3 krb5kdc[11852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079374, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:34.213711 11962 init.cc:377] Logged in from keytab as kudu/127.8.16.65@KRBTEST.COM (short username kudu)
I20260807 05:09:34.214504 11962 webserver.cc:460] Webserver started at http://127.8.16.65:38747/ using document root <none> and password file <none>
I20260807 05:09:34.215015 11962 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:34.215080 11962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:34.215273 11962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:34.217285 11962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "7f2eb6470375469f8e364c98295d9fa4"
format_stamp: "Formatted at 2026-08-07 05:09:34 on dist-test-slave-49h3"
server_key: "c5d6000a387458fe4ee1ae3e4d2fb382"
server_key_iv: "3285eb35bf890271c6aa55e1489a1a1b"
server_key_version: "encryptionkey@0"
I20260807 05:09:34.217868 11962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "7f2eb6470375469f8e364c98295d9fa4"
format_stamp: "Formatted at 2026-08-07 05:09:34 on dist-test-slave-49h3"
server_key: "c5d6000a387458fe4ee1ae3e4d2fb382"
server_key_iv: "3285eb35bf890271c6aa55e1489a1a1b"
server_key_version: "encryptionkey@0"
I20260807 05:09:34.221560 11962 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:09:34.223821 11977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:34.224963 11962 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:34.225072 11962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "7f2eb6470375469f8e364c98295d9fa4"
format_stamp: "Formatted at 2026-08-07 05:09:34 on dist-test-slave-49h3"
server_key: "c5d6000a387458fe4ee1ae3e4d2fb382"
server_key_iv: "3285eb35bf890271c6aa55e1489a1a1b"
server_key_version: "encryptionkey@0"
I20260807 05:09:34.225168 11962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:34.275604 11962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:34.278460 11962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:34.278682 11962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:34.279428 11962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:34.280493 11962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:34.280544 11962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:34.280583 11962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:34.280598 11962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:34.288969 11962 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:37627
I20260807 05:09:34.288998 12092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:37627 every 8 connection(s)
I20260807 05:09:34.289899 11962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:09:34.293843 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 11962
I20260807 05:09:34.293965 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:09:34.294234 8257 external_mini_cluster.cc:1428] Setting key effc2a20125e72d464cb8414670599a8
Aug 07 05:09:34 dist-test-slave-49h3 krb5kdc[11852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079374, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.16.66@KRBTEST.COM; defaulting to no policy
I20260807 05:09:34.303092 11939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:34.291620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:59987 (local address 127.8.16.126:42657)
0807 05:09:34.291764 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:34.291773 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:34.292553 (+ 780us) server_negotiation.cc:407] Connection header received
0807 05:09:34.293903 (+ 1350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:34.293906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:34.293962 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:34.294032 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:34.295271 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:34.295841 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:34.296987 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:34.297255 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:34.297339 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:34.299777 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:34.299781 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:34.299784 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:34.299807 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:34.301141 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:34.301659 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:34.301663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:34.301664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:34.301737 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:34.302045 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:34.302047 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:34.302048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:34.302218 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:34.302301 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:34.302796 (+ 495us) server_negotiation.cc:299] Negotiation successful
0807 05:09:34.302922 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 05:09:34.303880 12095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:34.291975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42657 (local address 127.8.16.65:59987)
0807 05:09:34.292412 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:34.292454 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:34.293697 (+ 1243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:34.294202 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:34.294209 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:34.294588 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:34.295135 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:34.295149 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:34.295956 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:34.295959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:34.296876 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:34.296885 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:34.297402 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:34.297406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:34.297597 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:34.298179 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:34.298195 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:34.299617 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:34.301246 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:34.301252 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:34.301274 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:34.301571 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:34.301814 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:34.301818 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:34.301820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:34.301952 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:34.302318 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:34.302324 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:34.302558 (+ 234us) client_negotiation.cc:769] Sending connection context
0807 05:09:34.302713 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 05:09:34.303031 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":115,"threads_started":1}
I20260807 05:09:34.304994 12093 heartbeater.cc:347] Connected to a master server at 127.8.16.126:42657
I20260807 05:09:34.305228 12093 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.66@KRBTEST.COM" created.
I20260807 05:09:34.305742 12093 heartbeater.cc:511] Master 127.8.16.126:42657 requested a full tablet report, sending...
I20260807 05:09:34.307161 11898 ts_manager.cc:194] Registered new tserver with Master: 7f2eb6470375469f8e364c98295d9fa4 (127.8.16.65:37627)
I20260807 05:09:34.308233 11898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:59987
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
I20260807 05:09:34.319700 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:42657
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:38291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:34.429059 12098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:34.429328 12098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:34.429370 12098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:34.433341 12098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:34.433409 12098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:34.433482 12098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:09:34.438761 12098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:38291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:42657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:34.439833 12098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:34.440860 12098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:34.450122 12106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:34.450134 12103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:34.450132 12104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:34.450198 12098 server_base.cc:1034] running on GCE node
I20260807 05:09:34.450778 12098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:34.451368 12098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:34.452510 12098 hybrid_clock.cc:648] HybridClock initialized: now 1786079374452486 us; error 43 us; skew 500 ppm
Aug 07 05:09:34 dist-test-slave-49h3 krb5kdc[11852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079374, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:34.455097 12098 init.cc:377] Logged in from keytab as kudu/127.8.16.66@KRBTEST.COM (short username kudu)
I20260807 05:09:34.455991 12098 webserver.cc:460] Webserver started at http://127.8.16.66:33573/ using document root <none> and password file <none>
I20260807 05:09:34.456516 12098 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:34.456579 12098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:34.456769 12098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:34.458781 12098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "ffdb1cf3310b46daab86fe0cd2584547"
format_stamp: "Formatted at 2026-08-07 05:09:34 on dist-test-slave-49h3"
server_key: "f337849d59ff758b3983da00bf3d9779"
server_key_iv: "85d1ae59b525ce2eb1baeead4674bcce"
server_key_version: "encryptionkey@0"
I20260807 05:09:34.459306 12098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "ffdb1cf3310b46daab86fe0cd2584547"
format_stamp: "Formatted at 2026-08-07 05:09:34 on dist-test-slave-49h3"
server_key: "f337849d59ff758b3983da00bf3d9779"
server_key_iv: "85d1ae59b525ce2eb1baeead4674bcce"
server_key_version: "encryptionkey@0"
I20260807 05:09:34.462884 12098 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:09:34.465106 12113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:34.466286 12098 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:34.466394 12098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "ffdb1cf3310b46daab86fe0cd2584547"
format_stamp: "Formatted at 2026-08-07 05:09:34 on dist-test-slave-49h3"
server_key: "f337849d59ff758b3983da00bf3d9779"
server_key_iv: "85d1ae59b525ce2eb1baeead4674bcce"
server_key_version: "encryptionkey@0"
I20260807 05:09:34.466490 12098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:34.488492 12098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:34.491420 12098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:34.491611 12098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:34.492197 12098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:34.493014 12098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:34.493062 12098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:34.493112 12098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:34.493144 12098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:34.501895 12098 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:44483
I20260807 05:09:34.501928 12230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:44483 every 8 connection(s)
I20260807 05:09:34.502818 12098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:09:34.506417 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 12098
I20260807 05:09:34.506539 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:09:34.506845 8257 external_mini_cluster.cc:1428] Setting key d91daeb773d55fa113a9f02a9517bd53
Aug 07 05:09:34 dist-test-slave-49h3 krb5kdc[11852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079374, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.16.67@KRBTEST.COM; defaulting to no policy
I20260807 05:09:34.516805 11939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:34.504520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:49003 (local address 127.8.16.126:42657)
0807 05:09:34.504649 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:34.504653 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:34.505470 (+ 817us) server_negotiation.cc:407] Connection header received
0807 05:09:34.507047 (+ 1577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:34.507050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:34.507098 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:34.507171 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:34.508534 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:34.509046 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:34.510568 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:34.510797 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:34.510887 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:34.513497 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:34.513500 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:34.513502 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:34.513527 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:34.514818 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:34.515289 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:34.515293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:34.515294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:34.515370 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:34.515683 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:34.515686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:34.515687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:34.515867 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:34.515952 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:34.516492 (+ 540us) server_negotiation.cc:299] Negotiation successful
0807 05:09:34.516622 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 05:09:34.517712 12233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:34.504879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42657 (local address 127.8.16.66:49003)
0807 05:09:34.505308 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:34.505350 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:34.506834 (+ 1484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:34.507354 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:34.507360 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:34.507778 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:34.508378 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:34.508399 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:34.509167 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:34.509177 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:34.510451 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:34.510461 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:34.510939 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:34.510943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:34.511109 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:34.511691 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:34.511706 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:34.513315 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:34.514922 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:34.514928 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:34.514945 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:34.515196 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:34.515449 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:34.515452 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:34.515454 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:34.515589 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:34.515965 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:34.515972 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:34.516257 (+ 285us) client_negotiation.cc:769] Sending connection context
0807 05:09:34.516429 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 05:09:34.516759 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":102,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.67@KRBTEST.COM" created.
I20260807 05:09:34.518824 12231 heartbeater.cc:347] Connected to a master server at 127.8.16.126:42657
I20260807 05:09:34.519066 12231 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:34.519551 12231 heartbeater.cc:511] Master 127.8.16.126:42657 requested a full tablet report, sending...
I20260807 05:09:34.520613 11898 ts_manager.cc:194] Registered new tserver with Master: ffdb1cf3310b46daab86fe0cd2584547 (127.8.16.66:44483)
I20260807 05:09:34.521178 11898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:49003
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
I20260807 05:09:34.533114 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:42657
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:38291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:34.642330 12236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:34.642618 12236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:34.642660 12236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:34.646749 12236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:34.646824 12236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:34.646884 12236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:09:34.652190 12236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:38291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:42657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:34.653304 12236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:34.654366 12236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:34.663558 12242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:34.663502 12241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:34.663712 12236 server_base.cc:1034] running on GCE node
W20260807 05:09:34.663530 12244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:34.664206 12236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:34.664800 12236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:34.665952 12236 hybrid_clock.cc:648] HybridClock initialized: now 1786079374665937 us; error 27 us; skew 500 ppm
Aug 07 05:09:34 dist-test-slave-49h3 krb5kdc[11852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079374, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:34.668681 12236 init.cc:377] Logged in from keytab as kudu/127.8.16.67@KRBTEST.COM (short username kudu)
I20260807 05:09:34.669730 12236 webserver.cc:460] Webserver started at http://127.8.16.67:34611/ using document root <none> and password file <none>
I20260807 05:09:34.670351 12236 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:34.670425 12236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:34.670634 12236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:34.672657 12236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "909f2a7c7ce74a069493968a2fd48029"
format_stamp: "Formatted at 2026-08-07 05:09:34 on dist-test-slave-49h3"
server_key: "7717680476dc628c37d1f0a0a158d48e"
server_key_iv: "1d23c2535e5b15d1350b940f7ac6266b"
server_key_version: "encryptionkey@0"
I20260807 05:09:34.673219 12236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "909f2a7c7ce74a069493968a2fd48029"
format_stamp: "Formatted at 2026-08-07 05:09:34 on dist-test-slave-49h3"
server_key: "7717680476dc628c37d1f0a0a158d48e"
server_key_iv: "1d23c2535e5b15d1350b940f7ac6266b"
server_key_version: "encryptionkey@0"
I20260807 05:09:34.677011 12236 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260807 05:09:34.679567 12252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:34.680857 12236 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:34.680991 12236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "909f2a7c7ce74a069493968a2fd48029"
format_stamp: "Formatted at 2026-08-07 05:09:34 on dist-test-slave-49h3"
server_key: "7717680476dc628c37d1f0a0a158d48e"
server_key_iv: "1d23c2535e5b15d1350b940f7ac6266b"
server_key_version: "encryptionkey@0"
I20260807 05:09:34.681106 12236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:34.702113 12236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:34.705094 12236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:34.705296 12236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:34.705924 12236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:34.706789 12236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:34.706838 12236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:34.706890 12236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:34.706919 12236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:34.715440 12236 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:41341
I20260807 05:09:34.715469 12368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:41341 every 8 connection(s)
I20260807 05:09:34.716303 12236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:09:34.719974 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 12236
I20260807 05:09:34.720083 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:09:34.720358 8257 external_mini_cluster.cc:1428] Setting key 5d3d422e5cf648a61dfbda8a8b72fea4
Aug 07 05:09:34 dist-test-slave-49h3 krb5kdc[11852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079374, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:34.729563 11939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:34.718088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:52323 (local address 127.8.16.126:42657)
0807 05:09:34.718215 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:34.718219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:34.719028 (+ 809us) server_negotiation.cc:407] Connection header received
0807 05:09:34.720444 (+ 1416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:34.720448 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:34.720501 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:34.720583 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:34.722025 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:34.722642 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:34.723713 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:34.723926 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:34.723998 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:34.726292 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:34.726296 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:34.726297 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:34.726321 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:34.727587 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:34.728060 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:34.728063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:34.728065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:34.728139 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:34.728466 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:34.728469 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:34.728471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:34.728671 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:34.728754 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:34.729249 (+ 495us) server_negotiation.cc:299] Negotiation successful
0807 05:09:34.729375 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 05:09:34.730357 12371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:34.718443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42657 (local address 127.8.16.67:52323)
0807 05:09:34.718880 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:34.718923 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:34.720204 (+ 1281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:34.720765 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:34.720774 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:34.721264 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:34.721899 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:34.721915 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:34.722750 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:34.722753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:34.723616 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:34.723625 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:34.724032 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:34.724034 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:34.724179 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:34.724682 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:34.724699 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:34.726139 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:34.727696 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:34.727701 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:34.727717 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:34.727969 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:34.728217 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:34.728220 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:34.728222 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:34.728379 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:34.728774 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:34.728781 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:34.729026 (+ 245us) client_negotiation.cc:769] Sending connection context
0807 05:09:34.729191 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 05:09:34.729546 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":115,"threads_started":1}
I20260807 05:09:34.731415 12369 heartbeater.cc:347] Connected to a master server at 127.8.16.126:42657
I20260807 05:09:34.731628 12369 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:34.732088 12369 heartbeater.cc:511] Master 127.8.16.126:42657 requested a full tablet report, sending...
I20260807 05:09:34.733057 11898 ts_manager.cc:194] Registered new tserver with Master: 909f2a7c7ce74a069493968a2fd48029 (127.8.16.67:41341)
I20260807 05:09:34.733639 11898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:52323
I20260807 05:09:34.733958 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 05:09:34 dist-test-slave-49h3 krb5kdc[11852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 05:09:34 dist-test-slave-49h3 krb5kdc[11852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.65@KRBTEST.COM
I20260807 05:09:34.759473 12376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:34.750518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44780 (local address 127.8.16.65:37627)
0807 05:09:34.750791 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:34.750809 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:34.750834 (+ 25us) server_negotiation.cc:407] Connection header received
0807 05:09:34.750884 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:34.750889 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:34.751070 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:34.751191 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:34.751791 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:34.752565 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:34.753362 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:34.753829 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:34.753939 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:34.756360 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:34.756371 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:34.756388 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:34.756416 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:34.758021 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:34.758408 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:34.758413 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:34.758414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:34.758489 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:34.758672 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:34.758674 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:34.758675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:34.758907 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:34.759053 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:34.759191 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 05:09:34.759319 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":92,"threads_started":1}
Aug 07 05:09:34 dist-test-slave-49h3 krb5kdc[11852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:34.769063 11939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:34.762283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44848 (local address 127.8.16.126:42657)
0807 05:09:34.762423 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:34.762427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:34.762440 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:09:34.762530 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:34.762532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:34.762572 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:34.762631 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:34.763109 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:34.763617 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:34.764293 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:34.764490 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:34.764555 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:34.766693 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:34.766696 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:34.766698 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:34.766723 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:34.767795 (+ 1072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:34.768117 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:34.768119 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:34.768120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:34.768195 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:34.768373 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:34.768375 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:34.768376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:34.768573 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:34.768669 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:34.768814 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 05:09:34.768906 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 05:09:34.769470 11898 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44848
I20260807 05:09:34.770040 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 11962
I20260807 05:09:34.776142 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 12098
I20260807 05:09:34.782644 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 12236
I20260807 05:09:34.789023 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 11866
2026-08-07T05:09:34Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2085 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:09:34 dist-test-slave-49h3 krb5kdc[12381](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:09:34 dist-test-slave-49h3 krb5kdc[12381](info): set up 2 sockets
Aug 07 05:09:34 dist-test-slave-49h3 krb5kdc[12381](info): commencing operation
krb5kdc: starting...
W20260807 05:09:35.769397 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.958s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:09:35 dist-test-slave-49h3 krb5kdc[12381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:09:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:09:35.886420 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:36143
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:37725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:36143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:35.997402 12394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:35.997722 12394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:35.997776 12394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:36.001817 12394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:36.001883 12394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:36.001900 12394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:36.001921 12394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:36.001951 12394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:36.007302 12394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:37725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:36143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:36143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:36.008411 12394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:36.009474 12394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:36.017751 12400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:36.017752 12402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:36.017769 12399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:36.017882 12394 server_base.cc:1034] running on GCE node
I20260807 05:09:36.018687 12394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:36.019783 12394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:36.020941 12394 hybrid_clock.cc:648] HybridClock initialized: now 1786079376020927 us; error 27 us; skew 500 ppm
Aug 07 05:09:36 dist-test-slave-49h3 krb5kdc[12381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079376, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:36.026561 12394 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:09:36.027314 12394 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:36.027387 12394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:36.027594 12394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:36.029724 12394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "7e425a41497043cab129ef7a231168cd"
format_stamp: "Formatted at 2026-08-07 05:09:36 on dist-test-slave-49h3"
server_key: "007b9dc018831555c0de45872570a269"
server_key_iv: "f2649d13a0fe7cec7bf3541a398ce89d"
server_key_version: "encryptionkey@0"
I20260807 05:09:36.030293 12394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "7e425a41497043cab129ef7a231168cd"
format_stamp: "Formatted at 2026-08-07 05:09:36 on dist-test-slave-49h3"
server_key: "007b9dc018831555c0de45872570a269"
server_key_iv: "f2649d13a0fe7cec7bf3541a398ce89d"
server_key_version: "encryptionkey@0"
I20260807 05:09:36.033999 12394 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 05:09:36.036316 12408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:36.037554 12394 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:36.037669 12394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "7e425a41497043cab129ef7a231168cd"
format_stamp: "Formatted at 2026-08-07 05:09:36 on dist-test-slave-49h3"
server_key: "007b9dc018831555c0de45872570a269"
server_key_iv: "f2649d13a0fe7cec7bf3541a398ce89d"
server_key_version: "encryptionkey@0"
I20260807 05:09:36.037773 12394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:36.056104 12394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:36.058856 12394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:36.059036 12394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:36.065690 12394 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:36143
I20260807 05:09:36.065930 12462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:36143 every 8 connection(s)
I20260807 05:09:36.066358 12394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:36.068961 12463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:36.072574 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 12394
I20260807 05:09:36.072693 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:09:36.072947 8257 external_mini_cluster.cc:1428] Setting key 2a51b7ea32a93f7feaf46fad0f5a8843
I20260807 05:09:36.074636 12463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd: Bootstrap starting.
I20260807 05:09:36.077266 12463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:36.078186 12463 log.cc:824] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:36.080319 12463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd: No bootstrap required, opened a new log
Aug 07 05:09:36 dist-test-slave-49h3 krb5kdc[12381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:36.084228 12463 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e425a41497043cab129ef7a231168cd" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36143 } }
I20260807 05:09:36.084520 12463 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:36.084578 12463 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e425a41497043cab129ef7a231168cd, State: Initialized, Role: FOLLOWER
I20260807 05:09:36.085047 12463 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e425a41497043cab129ef7a231168cd" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36143 } }
I20260807 05:09:36.085208 12463 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:09:36.085280 12463 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:09:36.085381 12463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:36.086536 12463 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e425a41497043cab129ef7a231168cd" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36143 } }
I20260807 05:09:36.086896 12463 leader_election.cc:302] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e425a41497043cab129ef7a231168cd; no voters:
I20260807 05:09:36.087217 12463 leader_election.cc:288] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:09:36.087347 12468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:36.087553 12468 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd [term 1 LEADER]: Becoming Leader. State: Replica: 7e425a41497043cab129ef7a231168cd, State: Running, Role: LEADER
I20260807 05:09:36.087719 12466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:36.074327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52586 (local address 127.8.16.126:36143)
0807 05:09:36.074745 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:36.074765 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:36.074807 (+ 42us) server_negotiation.cc:407] Connection header received
0807 05:09:36.075617 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:36.075639 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:36.076059 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:36.076422 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:36.077075 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.078572 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.079531 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.080200 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.080378 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:36.082762 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:36.082769 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:36.082788 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:36.082822 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:36.084683 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:36.085067 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:36.085072 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:36.085078 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:36.085215 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:36.085542 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:36.085546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:36.085547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:36.086028 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:36.086239 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:36.086437 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 05:09:36.086740 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":110,"threads_started":1}
I20260807 05:09:36.087868 12468 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e425a41497043cab129ef7a231168cd" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36143 } }
I20260807 05:09:36.088375 12463 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:36.089015 12470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e425a41497043cab129ef7a231168cd. Latest consensus state: current_term: 1 leader_uuid: "7e425a41497043cab129ef7a231168cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e425a41497043cab129ef7a231168cd" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36143 } } }
I20260807 05:09:36.089166 12470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:36.089990 12469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e425a41497043cab129ef7a231168cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e425a41497043cab129ef7a231168cd" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36143 } } }
I20260807 05:09:36.090088 12469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:36.090381 12477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:36.093742 12477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:36.099181 12477 catalog_manager.cc:1347] Generated new cluster ID: d1ea1d73cc26441f930419ff3e3f7b17
I20260807 05:09:36.099255 12477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:36.131628 12477 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:09:36.132933 12477 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:36.138172 12477 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd: Generated new TSK 0
I20260807 05:09:36.138998 12477 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
I20260807 05:09:36.169384 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36143
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:37725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:36.279014 12489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:36.279301 12489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:36.279366 12489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:36.283313 12489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:36.283376 12489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:36.283457 12489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:09:36.289731 12489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:37725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:36.290805 12489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:36.291813 12489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:36.300824 12498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:36.300823 12495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:36.300827 12496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:36.301151 12489 server_base.cc:1034] running on GCE node
I20260807 05:09:36.301573 12489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:36.302173 12489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:36.303321 12489 hybrid_clock.cc:648] HybridClock initialized: now 1786079376303307 us; error 31 us; skew 500 ppm
Aug 07 05:09:36 dist-test-slave-49h3 krb5kdc[12381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079376, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:36.306021 12489 init.cc:377] Logged in from keytab as kudu/127.8.16.65@KRBTEST.COM (short username kudu)
I20260807 05:09:36.306725 12489 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:36.306804 12489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:36.307008 12489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:36.308990 12489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "bd38b8f229aa45f2b4245d9c110e0848"
format_stamp: "Formatted at 2026-08-07 05:09:36 on dist-test-slave-49h3"
server_key: "71e0b84b3f60d4905999ecabd2beaa15"
server_key_iv: "06c1a8f0922cab1260e23981f15d8c4c"
server_key_version: "encryptionkey@0"
I20260807 05:09:36.309566 12489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "bd38b8f229aa45f2b4245d9c110e0848"
format_stamp: "Formatted at 2026-08-07 05:09:36 on dist-test-slave-49h3"
server_key: "71e0b84b3f60d4905999ecabd2beaa15"
server_key_iv: "06c1a8f0922cab1260e23981f15d8c4c"
server_key_version: "encryptionkey@0"
I20260807 05:09:36.313339 12489 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 05:09:36.315654 12504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:36.316811 12489 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:36.316916 12489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "bd38b8f229aa45f2b4245d9c110e0848"
format_stamp: "Formatted at 2026-08-07 05:09:36 on dist-test-slave-49h3"
server_key: "71e0b84b3f60d4905999ecabd2beaa15"
server_key_iv: "06c1a8f0922cab1260e23981f15d8c4c"
server_key_version: "encryptionkey@0"
I20260807 05:09:36.317006 12489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:36.326736 12489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:36.329329 12489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:36.329522 12489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:36.330092 12489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:36.330897 12489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:36.330945 12489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:36.330996 12489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:36.331024 12489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:36.339524 12489 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:39521
I20260807 05:09:36.339545 12618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:39521 every 8 connection(s)
I20260807 05:09:36.340118 12489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:09:36.345932 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 12489
I20260807 05:09:36.346083 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:09:36.346385 8257 external_mini_cluster.cc:1428] Setting key 5bca9261154afeba73b3c681f894803f
Aug 07 05:09:36 dist-test-slave-49h3 krb5kdc[12381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079376, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:36.353770 12466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:36.341807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:55401 (local address 127.8.16.126:36143)
0807 05:09:36.341933 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:36.341937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:36.342792 (+ 855us) server_negotiation.cc:407] Connection header received
0807 05:09:36.344018 (+ 1226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:36.344021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:36.344078 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:36.344150 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:36.345288 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.345978 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.347241 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.347560 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.347667 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:36.350200 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:36.350203 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:36.350205 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:36.350228 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:36.351679 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:36.352152 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:36.352156 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:36.352158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:36.352275 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:36.352618 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:36.352621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:36.352623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:36.352856 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:36.352963 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:36.353433 (+ 470us) server_negotiation.cc:299] Negotiation successful
0807 05:09:36.353593 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 05:09:36.354506 12621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:36.342198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36143 (local address 127.8.16.65:55401)
0807 05:09:36.342644 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:36.342697 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:36.343818 (+ 1121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:36.344325 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:36.344332 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:36.344689 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:36.345148 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:36.345161 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.346130 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.346133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:36.347130 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:36.347139 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.347702 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.347704 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:36.347887 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:36.348452 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:36.348471 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:36.350045 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:36.351789 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:36.351794 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:36.351806 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:36.352053 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:36.352361 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:36.352363 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:36.352365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:36.352523 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:36.352948 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:36.352953 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:36.353186 (+ 233us) client_negotiation.cc:769] Sending connection context
0807 05:09:36.353350 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 05:09:36.353643 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":102,"threads_started":1}
I20260807 05:09:36.355718 12619 heartbeater.cc:347] Connected to a master server at 127.8.16.126:36143
I20260807 05:09:36.355954 12619 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.8.16.66@KRBTEST.COM; defaulting to no policy
I20260807 05:09:36.356392 12619 heartbeater.cc:511] Master 127.8.16.126:36143 requested a full tablet report, sending...
I20260807 05:09:36.357894 12425 ts_manager.cc:194] Registered new tserver with Master: bd38b8f229aa45f2b4245d9c110e0848 (127.8.16.65:39521)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.66@KRBTEST.COM" created.
I20260807 05:09:36.359203 12425 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:55401
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
I20260807 05:09:36.376113 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36143
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:37725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:36.483723 12624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:36.483989 12624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:36.484050 12624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:36.488068 12624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:36.488140 12624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:36.488217 12624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:09:36.493475 12624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:37725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:36.494513 12624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:36.495503 12624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:36.504458 12629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:36.504458 12630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:36.504482 12632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:36.504745 12624 server_base.cc:1034] running on GCE node
I20260807 05:09:36.505165 12624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:36.505806 12624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:36.506968 12624 hybrid_clock.cc:648] HybridClock initialized: now 1786079376506947 us; error 38 us; skew 500 ppm
Aug 07 05:09:36 dist-test-slave-49h3 krb5kdc[12381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079376, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:36.509656 12624 init.cc:377] Logged in from keytab as kudu/127.8.16.66@KRBTEST.COM (short username kudu)
I20260807 05:09:36.510394 12624 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:36.510463 12624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:36.510659 12624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:36.512763 12624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "3c1fa15ed63d477a94666d9bd25283dc"
format_stamp: "Formatted at 2026-08-07 05:09:36 on dist-test-slave-49h3"
server_key: "d23631b30332599a30313c5b6be77308"
server_key_iv: "c99207088761edefa8d750688b6e6aa7"
server_key_version: "encryptionkey@0"
I20260807 05:09:36.513315 12624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "3c1fa15ed63d477a94666d9bd25283dc"
format_stamp: "Formatted at 2026-08-07 05:09:36 on dist-test-slave-49h3"
server_key: "d23631b30332599a30313c5b6be77308"
server_key_iv: "c99207088761edefa8d750688b6e6aa7"
server_key_version: "encryptionkey@0"
I20260807 05:09:36.516907 12624 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 05:09:36.519239 12639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:36.520439 12624 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:36.520543 12624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "3c1fa15ed63d477a94666d9bd25283dc"
format_stamp: "Formatted at 2026-08-07 05:09:36 on dist-test-slave-49h3"
server_key: "d23631b30332599a30313c5b6be77308"
server_key_iv: "c99207088761edefa8d750688b6e6aa7"
server_key_version: "encryptionkey@0"
I20260807 05:09:36.520668 12624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:36.535238 12624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:36.538018 12624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:36.538211 12624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:36.538774 12624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:36.539583 12624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:36.539630 12624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:36.539683 12624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:36.539712 12624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:36.548040 12624 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:39277
I20260807 05:09:36.548064 12754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:39277 every 8 connection(s)
I20260807 05:09:36.548624 12624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:09:36.552699 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 12624
I20260807 05:09:36.552819 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:09:36.553087 8257 external_mini_cluster.cc:1428] Setting key f81c1b99291873b01a1b167141cd5922
Aug 07 05:09:36 dist-test-slave-49h3 krb5kdc[12381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079376, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:36.561928 12466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:36.550271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:43609 (local address 127.8.16.126:36143)
0807 05:09:36.550430 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:36.550435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:36.551205 (+ 770us) server_negotiation.cc:407] Connection header received
0807 05:09:36.552382 (+ 1177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:36.552384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:36.552434 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:36.552523 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:36.553772 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.554454 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.555549 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.555839 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.555942 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:36.558269 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:36.558273 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:36.558275 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:36.558302 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:36.559794 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:36.560271 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:36.560275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:36.560277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:36.560373 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:36.560666 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:36.560669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:36.560670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:36.560914 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:36.561024 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:36.561609 (+ 585us) server_negotiation.cc:299] Negotiation successful
0807 05:09:36.561748 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 05:09:36.562686 12758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:36.550626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36143 (local address 127.8.16.66:43609)
0807 05:09:36.551051 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:36.551092 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:36.552190 (+ 1098us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:36.552664 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:36.552670 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:36.553061 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:36.553616 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:36.553634 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.554567 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.554570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:36.555429 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:36.555438 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.555964 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.555966 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:36.556101 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:36.556641 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:36.556657 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:36.558096 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:36.559908 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:36.559914 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:36.559926 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:36.560174 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:36.560456 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:36.560458 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:36.560460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:36.560587 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:36.561029 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:36.561035 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:36.561340 (+ 305us) client_negotiation.cc:769] Sending connection context
0807 05:09:36.561547 (+ 207us) client_negotiation.cc:240] Negotiation successful
0807 05:09:36.561818 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":102,"threads_started":1}
WARNING: no policy specified for kudu/127.8.16.67@KRBTEST.COM; defaulting to no policy
I20260807 05:09:36.563808 12755 heartbeater.cc:347] Connected to a master server at 127.8.16.126:36143
I20260807 05:09:36.564046 12755 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:36.564577 12755 heartbeater.cc:511] Master 127.8.16.126:36143 requested a full tablet report, sending...
I20260807 05:09:36.565616 12425 ts_manager.cc:194] Registered new tserver with Master: 3c1fa15ed63d477a94666d9bd25283dc (127.8.16.66:39277)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.67@KRBTEST.COM" created.
I20260807 05:09:36.566262 12425 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:43609
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
I20260807 05:09:36.580782 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36143
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:37725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:36.690454 12761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:36.690737 12761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:36.690790 12761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:36.694825 12761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:36.694890 12761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:36.694965 12761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:09:36.700275 12761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:37725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:36.701341 12761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:36.702358 12761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:36.711447 12767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:36.711447 12769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:36.711501 12766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:36.711495 12761 server_base.cc:1034] running on GCE node
I20260807 05:09:36.712069 12761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:36.712755 12761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:36.713915 12761 hybrid_clock.cc:648] HybridClock initialized: now 1786079376713889 us; error 42 us; skew 500 ppm
Aug 07 05:09:36 dist-test-slave-49h3 krb5kdc[12381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079376, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:36.716718 12761 init.cc:377] Logged in from keytab as kudu/127.8.16.67@KRBTEST.COM (short username kudu)
I20260807 05:09:36.717412 12761 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:36.717509 12761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:36.717711 12761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:36.719704 12761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "b8822969a60842dd9b1737bc505e7289"
format_stamp: "Formatted at 2026-08-07 05:09:36 on dist-test-slave-49h3"
server_key: "99680a1648ea4d01a992fc41fe4403d8"
server_key_iv: "cbcd4b8cd266ac1f65420acdeed85ffe"
server_key_version: "encryptionkey@0"
I20260807 05:09:36.720293 12761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "b8822969a60842dd9b1737bc505e7289"
format_stamp: "Formatted at 2026-08-07 05:09:36 on dist-test-slave-49h3"
server_key: "99680a1648ea4d01a992fc41fe4403d8"
server_key_iv: "cbcd4b8cd266ac1f65420acdeed85ffe"
server_key_version: "encryptionkey@0"
I20260807 05:09:36.723886 12761 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:09:36.726348 12775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:36.727572 12761 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:36.727680 12761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "b8822969a60842dd9b1737bc505e7289"
format_stamp: "Formatted at 2026-08-07 05:09:36 on dist-test-slave-49h3"
server_key: "99680a1648ea4d01a992fc41fe4403d8"
server_key_iv: "cbcd4b8cd266ac1f65420acdeed85ffe"
server_key_version: "encryptionkey@0"
I20260807 05:09:36.727774 12761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:36.753060 12761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:36.755971 12761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:36.756151 12761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:36.756731 12761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:36.757572 12761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:36.757618 12761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:36.757661 12761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:36.757690 12761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:36.765995 12761 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:40259
I20260807 05:09:36.766037 12890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:40259 every 8 connection(s)
I20260807 05:09:36.766613 12761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:09:36.767405 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 12761
I20260807 05:09:36.767500 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:09:36.767776 8257 external_mini_cluster.cc:1428] Setting key b342203c62c0672b83b8d66bd46e29f2
Aug 07 05:09:36 dist-test-slave-49h3 krb5kdc[12381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079376, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:36.779683 12466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:36.769145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:55731 (local address 127.8.16.126:36143)
0807 05:09:36.769294 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:36.769299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:36.770297 (+ 998us) server_negotiation.cc:407] Connection header received
0807 05:09:36.771481 (+ 1184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:36.771483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:36.771529 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:36.771594 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:36.772663 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.773182 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.774293 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.774500 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.774574 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:36.776842 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:36.776846 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:36.776848 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:36.776873 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:36.778024 (+ 1151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:36.778455 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:36.778457 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:36.778458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:36.778528 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:36.778782 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:36.778785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:36.778786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:36.778959 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:36.779033 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:36.779444 (+ 411us) server_negotiation.cc:299] Negotiation successful
0807 05:09:36.779548 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 05:09:36.780575 12894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:36.769684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36143 (local address 127.8.16.67:55731)
0807 05:09:36.770164 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:36.770207 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:36.771308 (+ 1101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:36.771724 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:36.771730 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:36.772082 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:36.772556 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:36.772570 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.773271 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.773274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:36.774200 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:36.774210 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.774612 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.774615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:36.774756 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:36.775272 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:36.775288 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:36.776689 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:36.778126 (+ 1437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:36.778130 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:36.778143 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:36.778385 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:36.778590 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:36.778592 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:36.778593 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:36.778712 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:36.779031 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:36.779035 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:36.779263 (+ 228us) client_negotiation.cc:769] Sending connection context
0807 05:09:36.779409 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 05:09:36.779651 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":139,"threads_started":1}
I20260807 05:09:36.781711 12891 heartbeater.cc:347] Connected to a master server at 127.8.16.126:36143
I20260807 05:09:36.781953 12891 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:36.782362 12891 heartbeater.cc:511] Master 127.8.16.126:36143 requested a full tablet report, sending...
I20260807 05:09:36.783262 12425 ts_manager.cc:194] Registered new tserver with Master: b8822969a60842dd9b1737bc505e7289 (127.8.16.67:40259)
I20260807 05:09:36.783799 12425 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:55731
I20260807 05:09:36.793473 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:09:36.801661 12466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:36.795324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52592 (local address 127.8.16.126:36143)
0807 05:09:36.795431 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:36.795434 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:36.795552 (+ 118us) server_negotiation.cc:407] Connection header received
0807 05:09:36.795670 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:36.795673 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:36.795711 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:36.795766 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:36.796390 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.796889 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.797676 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.797888 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.797962 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:36.799186 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:36.799189 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:36.799191 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:36.799212 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:36.800350 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:36.800687 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:36.800689 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:36.800690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:36.800757 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:36.800930 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:36.800931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:36.800933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:36.801130 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:36.801241 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:36.801408 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 05:09:36.801538 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 05:09:36.804250 12425 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52592:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:09:36.807287 12425 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:09:36.821141 12904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:36.815106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42776 (local address 127.8.16.65:39521)
0807 05:09:36.815457 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:36.815470 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:36.815498 (+ 28us) server_negotiation.cc:407] Connection header received
0807 05:09:36.815578 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:36.815583 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:36.816240 (+ 657us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:36.816766 (+ 526us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:36.816919 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.818083 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.819582 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.820461 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.820563 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:36.820702 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:36.820879 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 05:09:36.820976 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":137,"threads_started":1}
I20260807 05:09:36.821142 12905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:36.815661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:40259 (local address 127.0.0.1:44966)
0807 05:09:36.815895 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:36.815908 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:36.815986 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:36.817217 (+ 1231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:36.817219 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:36.817237 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:36.817318 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:36.817323 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.818529 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.818533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:36.819572 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:36.819583 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.820763 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.820767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:36.820849 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:36.820872 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:09:36.820935 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 05:09:36.820998 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":105,"threads_started":1}
I20260807 05:09:36.821611 12907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:36.815817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44966 (local address 127.8.16.67:40259)
0807 05:09:36.816811 (+ 994us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:36.816826 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:36.816845 (+ 19us) server_negotiation.cc:407] Connection header received
0807 05:09:36.816914 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:36.816918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:36.817083 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:36.817243 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:36.817439 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.818410 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.819691 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.820461 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.820563 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:36.820702 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:36.821345 (+ 643us) server_negotiation.cc:299] Negotiation successful
0807 05:09:36.821436 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":905,"thread_start_us":152,"threads_started":1}
I20260807 05:09:36.821638 12908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:36.816088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41854 (local address 127.8.16.66:39277)
0807 05:09:36.816442 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:36.816454 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:36.816568 (+ 114us) server_negotiation.cc:407] Connection header received
0807 05:09:36.816673 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:36.816676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:36.816814 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:36.816937 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:36.817149 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.818019 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.820008 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.820785 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.820875 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:36.821000 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:36.821382 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 05:09:36.821500 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":140,"threads_started":1}
I20260807 05:09:36.821156 12903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:36.814964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:39521 (local address 127.0.0.1:42776)
0807 05:09:36.815280 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:36.815326 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:36.815446 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:36.816424 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:36.816430 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:36.816453 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:36.816608 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:36.816614 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.818193 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.818196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:36.819472 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:36.819481 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.820570 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.820575 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:36.820682 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:36.820723 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 05:09:36.820843 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 05:09:36.820965 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":51,"thread_start_us":90,"threads_started":1}
I20260807 05:09:36.822069 12906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:36.815679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:39277 (local address 127.0.0.1:41854)
0807 05:09:36.816482 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:36.816495 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:36.816581 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:36.816935 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:36.816938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:36.816954 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:36.817058 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:36.817065 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.818128 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.818131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:36.819903 (+ 1772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:36.819910 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.821062 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.821065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:36.821154 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:36.821176 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 05:09:36.821241 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 05:09:36.821309 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"mutex_wait_us":102,"thread_start_us":98,"threads_started":1}
I20260807 05:09:36.824523 12553 tablet_service.cc:1467] Processing CreateTablet for tablet 305e45f37486438eb3926baa8300d224 (DEFAULT_TABLE table=test-table [id=5212df54fbd447c996289eba72cf031a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:36.824524 12825 tablet_service.cc:1467] Processing CreateTablet for tablet 305e45f37486438eb3926baa8300d224 (DEFAULT_TABLE table=test-table [id=5212df54fbd447c996289eba72cf031a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:36.824525 12689 tablet_service.cc:1467] Processing CreateTablet for tablet 305e45f37486438eb3926baa8300d224 (DEFAULT_TABLE table=test-table [id=5212df54fbd447c996289eba72cf031a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:36.825551 12553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 305e45f37486438eb3926baa8300d224. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:36.825568 12689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 305e45f37486438eb3926baa8300d224. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:36.825577 12825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 305e45f37486438eb3926baa8300d224. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:36.832154 12909 tablet_bootstrap.cc:492] T 305e45f37486438eb3926baa8300d224 P bd38b8f229aa45f2b4245d9c110e0848: Bootstrap starting.
I20260807 05:09:36.833011 12910 tablet_bootstrap.cc:492] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289: Bootstrap starting.
I20260807 05:09:36.833765 12911 tablet_bootstrap.cc:492] T 305e45f37486438eb3926baa8300d224 P 3c1fa15ed63d477a94666d9bd25283dc: Bootstrap starting.
I20260807 05:09:36.834383 12909 tablet_bootstrap.cc:654] T 305e45f37486438eb3926baa8300d224 P bd38b8f229aa45f2b4245d9c110e0848: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:36.834945 12910 tablet_bootstrap.cc:654] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:36.835425 12909 log.cc:824] T 305e45f37486438eb3926baa8300d224 P bd38b8f229aa45f2b4245d9c110e0848: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:36.835682 12911 tablet_bootstrap.cc:654] T 305e45f37486438eb3926baa8300d224 P 3c1fa15ed63d477a94666d9bd25283dc: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:36.836118 12910 log.cc:824] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:36.836760 12911 log.cc:824] T 305e45f37486438eb3926baa8300d224 P 3c1fa15ed63d477a94666d9bd25283dc: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:36.837406 12909 tablet_bootstrap.cc:492] T 305e45f37486438eb3926baa8300d224 P bd38b8f229aa45f2b4245d9c110e0848: No bootstrap required, opened a new log
I20260807 05:09:36.837621 12909 ts_tablet_manager.cc:1397] T 305e45f37486438eb3926baa8300d224 P bd38b8f229aa45f2b4245d9c110e0848: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260807 05:09:36.838151 12910 tablet_bootstrap.cc:492] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289: No bootstrap required, opened a new log
I20260807 05:09:36.838332 12910 ts_tablet_manager.cc:1397] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:09:36.841544 12909 raft_consensus.cc:348] T 305e45f37486438eb3926baa8300d224 P bd38b8f229aa45f2b4245d9c110e0848 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd38b8f229aa45f2b4245d9c110e0848" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39521 } } peers { permanent_uuid: "b8822969a60842dd9b1737bc505e7289" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 40259 } } peers { permanent_uuid: "3c1fa15ed63d477a94666d9bd25283dc" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 39277 } }
I20260807 05:09:36.841843 12909 raft_consensus.cc:374] T 305e45f37486438eb3926baa8300d224 P bd38b8f229aa45f2b4245d9c110e0848 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:36.841907 12909 raft_consensus.cc:729] T 305e45f37486438eb3926baa8300d224 P bd38b8f229aa45f2b4245d9c110e0848 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd38b8f229aa45f2b4245d9c110e0848, State: Initialized, Role: FOLLOWER
I20260807 05:09:36.841922 12910 raft_consensus.cc:348] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd38b8f229aa45f2b4245d9c110e0848" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39521 } } peers { permanent_uuid: "b8822969a60842dd9b1737bc505e7289" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 40259 } } peers { permanent_uuid: "3c1fa15ed63d477a94666d9bd25283dc" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 39277 } }
I20260807 05:09:36.842175 12910 raft_consensus.cc:374] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:36.842250 12910 raft_consensus.cc:729] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8822969a60842dd9b1737bc505e7289, State: Initialized, Role: FOLLOWER
I20260807 05:09:36.842478 12909 consensus_queue.cc:260] T 305e45f37486438eb3926baa8300d224 P bd38b8f229aa45f2b4245d9c110e0848 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd38b8f229aa45f2b4245d9c110e0848" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39521 } } peers { permanent_uuid: "b8822969a60842dd9b1737bc505e7289" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 40259 } } peers { permanent_uuid: "3c1fa15ed63d477a94666d9bd25283dc" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 39277 } }
I20260807 05:09:36.842617 12910 consensus_queue.cc:260] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd38b8f229aa45f2b4245d9c110e0848" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39521 } } peers { permanent_uuid: "b8822969a60842dd9b1737bc505e7289" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 40259 } } peers { permanent_uuid: "3c1fa15ed63d477a94666d9bd25283dc" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 39277 } }
I20260807 05:09:36.843159 12619 heartbeater.cc:503] Master 127.8.16.126:36143 was elected leader, sending a full tablet report...
I20260807 05:09:36.843524 12910 ts_tablet_manager.cc:1428] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 05:09:36.843524 12909 ts_tablet_manager.cc:1428] T 305e45f37486438eb3926baa8300d224 P bd38b8f229aa45f2b4245d9c110e0848: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260807 05:09:36.843691 12911 tablet_bootstrap.cc:492] T 305e45f37486438eb3926baa8300d224 P 3c1fa15ed63d477a94666d9bd25283dc: No bootstrap required, opened a new log
I20260807 05:09:36.843871 12911 ts_tablet_manager.cc:1397] T 305e45f37486438eb3926baa8300d224 P 3c1fa15ed63d477a94666d9bd25283dc: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260807 05:09:36.843899 12891 heartbeater.cc:503] Master 127.8.16.126:36143 was elected leader, sending a full tablet report...
I20260807 05:09:36.847448 12911 raft_consensus.cc:348] T 305e45f37486438eb3926baa8300d224 P 3c1fa15ed63d477a94666d9bd25283dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd38b8f229aa45f2b4245d9c110e0848" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39521 } } peers { permanent_uuid: "b8822969a60842dd9b1737bc505e7289" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 40259 } } peers { permanent_uuid: "3c1fa15ed63d477a94666d9bd25283dc" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 39277 } }
I20260807 05:09:36.847671 12911 raft_consensus.cc:374] T 305e45f37486438eb3926baa8300d224 P 3c1fa15ed63d477a94666d9bd25283dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:36.847723 12911 raft_consensus.cc:729] T 305e45f37486438eb3926baa8300d224 P 3c1fa15ed63d477a94666d9bd25283dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c1fa15ed63d477a94666d9bd25283dc, State: Initialized, Role: FOLLOWER
I20260807 05:09:36.848140 12911 consensus_queue.cc:260] T 305e45f37486438eb3926baa8300d224 P 3c1fa15ed63d477a94666d9bd25283dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd38b8f229aa45f2b4245d9c110e0848" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39521 } } peers { permanent_uuid: "b8822969a60842dd9b1737bc505e7289" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 40259 } } peers { permanent_uuid: "3c1fa15ed63d477a94666d9bd25283dc" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 39277 } }
I20260807 05:09:36.848785 12755 heartbeater.cc:503] Master 127.8.16.126:36143 was elected leader, sending a full tablet report...
I20260807 05:09:36.849085 12911 ts_tablet_manager.cc:1428] T 305e45f37486438eb3926baa8300d224 P 3c1fa15ed63d477a94666d9bd25283dc: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 05:09:36.881989 12916 raft_consensus.cc:482] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:09:36.882210 12916 raft_consensus.cc:504] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd38b8f229aa45f2b4245d9c110e0848" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39521 } } peers { permanent_uuid: "b8822969a60842dd9b1737bc505e7289" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 40259 } } peers { permanent_uuid: "3c1fa15ed63d477a94666d9bd25283dc" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 39277 } }
I20260807 05:09:36.883260 12916 leader_election.cc:288] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd38b8f229aa45f2b4245d9c110e0848 (127.8.16.65:39521), 3c1fa15ed63d477a94666d9bd25283dc (127.8.16.66:39277)
I20260807 05:09:36.887858 12904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:36.883592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:40631 (local address 127.8.16.65:39521)
0807 05:09:36.883707 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:36.883711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:36.883724 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:09:36.883869 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:36.883873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:36.883920 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:36.883989 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:36.884243 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.884992 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.886562 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.887223 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.887318 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:36.887380 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:36.887651 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 05:09:36.887736 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 05:09:36.887858 12908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:36.883708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:40525 (local address 127.8.16.66:39277)
0807 05:09:36.883843 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:36.883848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:36.884000 (+ 152us) server_negotiation.cc:407] Connection header received
0807 05:09:36.884104 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:36.884107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:36.884148 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:36.884215 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:36.884429 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.885133 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.886562 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.887196 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.887319 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:36.887380 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:36.887651 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 05:09:36.887732 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 05:09:36.887863 12894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:36.883477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:39521 (local address 127.8.16.67:40631)
0807 05:09:36.883656 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:36.883671 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:36.883756 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:36.884013 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:36.884016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:36.884029 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:36.884145 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:36.884151 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.885094 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.885097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:36.886408 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:36.886420 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.887318 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.887321 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:36.887488 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:36.887548 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 05:09:36.887618 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 05:09:36.887703 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":68}
I20260807 05:09:36.887876 12918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:36.883497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:39277 (local address 127.8.16.67:40525)
0807 05:09:36.883914 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:36.883925 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:36.884012 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:36.884236 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:36.884239 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:36.884256 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:36.884337 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:36.884341 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.885219 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.885222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:36.886428 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:36.886434 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.887318 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.887321 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:36.887485 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:36.887546 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 05:09:36.887618 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 05:09:36.887703 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":91,"thread_start_us":122,"threads_started":1}
I20260807 05:09:36.888518 12573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "305e45f37486438eb3926baa8300d224" candidate_uuid: "b8822969a60842dd9b1737bc505e7289" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd38b8f229aa45f2b4245d9c110e0848" is_pre_election: true
I20260807 05:09:36.888695 12709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "305e45f37486438eb3926baa8300d224" candidate_uuid: "b8822969a60842dd9b1737bc505e7289" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c1fa15ed63d477a94666d9bd25283dc" is_pre_election: true
I20260807 05:09:36.888849 12573 raft_consensus.cc:2454] T 305e45f37486438eb3926baa8300d224 P bd38b8f229aa45f2b4245d9c110e0848 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8822969a60842dd9b1737bc505e7289 in term 0.
I20260807 05:09:36.888983 12709 raft_consensus.cc:2454] T 305e45f37486438eb3926baa8300d224 P 3c1fa15ed63d477a94666d9bd25283dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8822969a60842dd9b1737bc505e7289 in term 0.
I20260807 05:09:36.889426 12779 leader_election.cc:302] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3c1fa15ed63d477a94666d9bd25283dc, b8822969a60842dd9b1737bc505e7289, bd38b8f229aa45f2b4245d9c110e0848; no voters:
I20260807 05:09:36.889755 12916 raft_consensus.cc:2781] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:09:36.889839 12916 raft_consensus.cc:482] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:09:36.889889 12916 raft_consensus.cc:3037] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:36.890977 12916 raft_consensus.cc:504] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd38b8f229aa45f2b4245d9c110e0848" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39521 } } peers { permanent_uuid: "b8822969a60842dd9b1737bc505e7289" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 40259 } } peers { permanent_uuid: "3c1fa15ed63d477a94666d9bd25283dc" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 39277 } }
I20260807 05:09:36.891299 12916 leader_election.cc:288] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289 [CANDIDATE]: Term 1 election: Requested vote from peers bd38b8f229aa45f2b4245d9c110e0848 (127.8.16.65:39521), 3c1fa15ed63d477a94666d9bd25283dc (127.8.16.66:39277)
I20260807 05:09:36.891676 12709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "305e45f37486438eb3926baa8300d224" candidate_uuid: "b8822969a60842dd9b1737bc505e7289" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c1fa15ed63d477a94666d9bd25283dc"
I20260807 05:09:36.891690 12573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "305e45f37486438eb3926baa8300d224" candidate_uuid: "b8822969a60842dd9b1737bc505e7289" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd38b8f229aa45f2b4245d9c110e0848"
I20260807 05:09:36.891795 12709 raft_consensus.cc:3037] T 305e45f37486438eb3926baa8300d224 P 3c1fa15ed63d477a94666d9bd25283dc [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:36.891801 12573 raft_consensus.cc:3037] T 305e45f37486438eb3926baa8300d224 P bd38b8f229aa45f2b4245d9c110e0848 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:36.892827 12573 raft_consensus.cc:2454] T 305e45f37486438eb3926baa8300d224 P bd38b8f229aa45f2b4245d9c110e0848 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8822969a60842dd9b1737bc505e7289 in term 1.
I20260807 05:09:36.892880 12709 raft_consensus.cc:2454] T 305e45f37486438eb3926baa8300d224 P 3c1fa15ed63d477a94666d9bd25283dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8822969a60842dd9b1737bc505e7289 in term 1.
I20260807 05:09:36.893142 12776 leader_election.cc:302] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8822969a60842dd9b1737bc505e7289, bd38b8f229aa45f2b4245d9c110e0848; no voters:
I20260807 05:09:36.893301 12916 raft_consensus.cc:2781] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:36.893567 12916 raft_consensus.cc:686] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289 [term 1 LEADER]: Becoming Leader. State: Replica: b8822969a60842dd9b1737bc505e7289, State: Running, Role: LEADER
I20260807 05:09:36.894102 12916 consensus_queue.cc:237] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd38b8f229aa45f2b4245d9c110e0848" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39521 } } peers { permanent_uuid: "b8822969a60842dd9b1737bc505e7289" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 40259 } } peers { permanent_uuid: "3c1fa15ed63d477a94666d9bd25283dc" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 39277 } }
I20260807 05:09:36.897189 12425 catalog_manager.cc:5495] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8822969a60842dd9b1737bc505e7289 (127.8.16.67). New cstate: current_term: 1 leader_uuid: "b8822969a60842dd9b1737bc505e7289" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd38b8f229aa45f2b4245d9c110e0848" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8822969a60842dd9b1737bc505e7289" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 40259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c1fa15ed63d477a94666d9bd25283dc" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 39277 } health_report { overall_health: UNKNOWN } } }
I20260807 05:09:36.923679 12907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:36.920389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44970 (local address 127.8.16.67:40259)
0807 05:09:36.920503 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:36.920508 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:36.920524 (+ 16us) server_negotiation.cc:407] Connection header received
0807 05:09:36.920671 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:36.920673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:36.920729 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:36.920813 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:36.921061 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.921623 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.922416 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:36.922619 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:36.922686 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:36.922910 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:36.923316 (+ 406us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:36.923405 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:36.923461 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 05:09:36.923517 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 05:09:36.929302 12709 raft_consensus.cc:1261] T 305e45f37486438eb3926baa8300d224 P 3c1fa15ed63d477a94666d9bd25283dc [term 1 FOLLOWER]: Refusing update from remote peer b8822969a60842dd9b1737bc505e7289: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:36.929302 12573 raft_consensus.cc:1261] T 305e45f37486438eb3926baa8300d224 P bd38b8f229aa45f2b4245d9c110e0848 [term 1 FOLLOWER]: Refusing update from remote peer b8822969a60842dd9b1737bc505e7289: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:36.930071 12919 consensus_queue.cc:1035] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd38b8f229aa45f2b4245d9c110e0848" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:36.930265 12916 consensus_queue.cc:1035] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c1fa15ed63d477a94666d9bd25283dc" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 39277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:36.938226 12923 mvcc.cc:205] Tried to move back new op lower bound from 7315781127895445504 to 7315781127760838656. Current Snapshot: MvccSnapshot[applied={T|T < 7315781127895445504}]
I20260807 05:09:36.946331 12425 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:36.946519 12425 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:36.949012 12425 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e425a41497043cab129ef7a231168cd: Sending DeleteTablet for 3 replicas of tablet 305e45f37486438eb3926baa8300d224
I20260807 05:09:36.949842 12689 tablet_service.cc:1514] Processing DeleteTablet for tablet 305e45f37486438eb3926baa8300d224 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:36 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:41854
I20260807 05:09:36.949847 12825 tablet_service.cc:1514] Processing DeleteTablet for tablet 305e45f37486438eb3926baa8300d224 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:36 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:44966
I20260807 05:09:36.950342 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 12489
I20260807 05:09:36.950587 12932 tablet_replica.cc:330] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289: stopping tablet replica
I20260807 05:09:36.950937 12932 raft_consensus.cc:2229] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:09:36.951354 12932 raft_consensus.cc:2258] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:36.951861 12933 tablet_replica.cc:330] T 305e45f37486438eb3926baa8300d224 P 3c1fa15ed63d477a94666d9bd25283dc: stopping tablet replica
I20260807 05:09:36.952200 12933 raft_consensus.cc:2229] T 305e45f37486438eb3926baa8300d224 P 3c1fa15ed63d477a94666d9bd25283dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:36.952553 12933 raft_consensus.cc:2258] T 305e45f37486438eb3926baa8300d224 P 3c1fa15ed63d477a94666d9bd25283dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:36.953291 12932 ts_tablet_manager.cc:1905] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:36.954080 12933 ts_tablet_manager.cc:1905] T 305e45f37486438eb3926baa8300d224 P 3c1fa15ed63d477a94666d9bd25283dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:36.956073 12932 ts_tablet_manager.cc:1918] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:09:36.956180 12932 log.cc:1196] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/305e45f37486438eb3926baa8300d224
I20260807 05:09:36.956519 12932 ts_tablet_manager.cc:1939] T 305e45f37486438eb3926baa8300d224 P b8822969a60842dd9b1737bc505e7289: Deleting consensus metadata
I20260807 05:09:36.956949 12933 ts_tablet_manager.cc:1918] T 305e45f37486438eb3926baa8300d224 P 3c1fa15ed63d477a94666d9bd25283dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:09:36.957105 12933 log.cc:1196] T 305e45f37486438eb3926baa8300d224 P 3c1fa15ed63d477a94666d9bd25283dc: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079354071073-8257-0/minicluster-data/ts-1/wal/wals/305e45f37486438eb3926baa8300d224
I20260807 05:09:36.957505 12933 ts_tablet_manager.cc:1939] T 305e45f37486438eb3926baa8300d224 P 3c1fa15ed63d477a94666d9bd25283dc: Deleting consensus metadata
I20260807 05:09:36.957829 12409 catalog_manager.cc:4841] TS b8822969a60842dd9b1737bc505e7289 (127.8.16.67:40259): tablet 305e45f37486438eb3926baa8300d224 (table test-table [id=5212df54fbd447c996289eba72cf031a]) successfully deleted
I20260807 05:09:36.958961 12411 catalog_manager.cc:4841] TS 3c1fa15ed63d477a94666d9bd25283dc (127.8.16.66:39277): tablet 305e45f37486438eb3926baa8300d224 (table test-table [id=5212df54fbd447c996289eba72cf031a]) successfully deleted
W20260807 05:09:36.959638 12409 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.16.65:39521 (error 108)
I20260807 05:09:36.959872 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 12624
I20260807 05:09:36.961702 12906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:36.960341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:39521 (local address 127.0.0.1:42792)
0807 05:09:36.961524 (+ 1183us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:36.961608 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.65:39521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1058}
W20260807 05:09:36.962678 12409 catalog_manager.cc:4584] TS bd38b8f229aa45f2b4245d9c110e0848 (127.8.16.65:39521): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 305e45f37486438eb3926baa8300d224: Network error: Client connection negotiation failed: client connection to 127.8.16.65:39521: connect: Connection refused (error 111)
I20260807 05:09:36.966409 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 12761
I20260807 05:09:36.972800 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 12394
2026-08-07T05:09:36Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2183 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T05:09:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:36Z Disabled control of system clock
I20260807 05:09:37.002363 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:36283
--webserver_interface=127.8.16.126
--webserver_port=0
--builtin_ntp_servers=127.8.16.84:46121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:36283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 05:09:37.109781 12938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:37.110054 12938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:37.110092 12938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:37.114042 12938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:37.114110 12938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:37.114125 12938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:37.114138 12938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:37.114152 12938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:37.118907 12938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:36283
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:36283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:37.119947 12938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:37.121269 12938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:37.129606 12943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:37.129721 12938 server_base.cc:1034] running on GCE node
W20260807 05:09:37.129627 12944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:37.129606 12946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:37.130401 12938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:37.131512 12938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:37.132683 12938 hybrid_clock.cc:648] HybridClock initialized: now 1786079377132667 us; error 39 us; skew 500 ppm
I20260807 05:09:37.134584 12938 webserver.cc:460] Webserver started at http://127.8.16.126:42615/ using document root <none> and password file <none>
I20260807 05:09:37.135160 12938 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:37.135231 12938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:37.135430 12938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:37.137586 12938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "e4d7960f76434b7b82f3a0eb85d64bc8"
format_stamp: "Formatted at 2026-08-07 05:09:37 on dist-test-slave-49h3"
server_key: "9d8e3a17f7d355a9bb1f53f2106ef31e"
server_key_iv: "9c82c4a4d67dbb3acebdbe8f988c29d9"
server_key_version: "encryptionkey@0"
I20260807 05:09:37.138142 12938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "e4d7960f76434b7b82f3a0eb85d64bc8"
format_stamp: "Formatted at 2026-08-07 05:09:37 on dist-test-slave-49h3"
server_key: "9d8e3a17f7d355a9bb1f53f2106ef31e"
server_key_iv: "9c82c4a4d67dbb3acebdbe8f988c29d9"
server_key_version: "encryptionkey@0"
I20260807 05:09:37.141675 12938 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 05:09:37.143956 12952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:37.145234 12938 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:37.145378 12938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "e4d7960f76434b7b82f3a0eb85d64bc8"
format_stamp: "Formatted at 2026-08-07 05:09:37 on dist-test-slave-49h3"
server_key: "9d8e3a17f7d355a9bb1f53f2106ef31e"
server_key_iv: "9c82c4a4d67dbb3acebdbe8f988c29d9"
server_key_version: "encryptionkey@0"
I20260807 05:09:37.145515 12938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:37.150137 12938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:37.150744 12938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:37.150906 12938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:37.157353 12938 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:36283
I20260807 05:09:37.157366 13004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:36283 every 8 connection(s)
I20260807 05:09:37.158290 12938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:37.161412 13006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:37.167200 13006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8: Bootstrap starting.
I20260807 05:09:37.167429 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 12938
I20260807 05:09:37.167546 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:09:37.167780 8257 external_mini_cluster.cc:1428] Setting key b7a4103dddf97f83913579d83a44d934
I20260807 05:09:37.169773 13006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:37.170487 13006 log.cc:824] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:37.172365 13006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8: No bootstrap required, opened a new log
I20260807 05:09:37.173168 13009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:37.169075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52526 (local address 127.8.16.126:36283)
0807 05:09:37.169764 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:37.169788 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:37.169823 (+ 35us) server_negotiation.cc:407] Connection header received
0807 05:09:37.170467 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:37.170484 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:37.170853 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:37.171149 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:37.171935 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:37.171940 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:37.171951 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:09:37.171960 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:37.172014 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:09:37.172030 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:37.172122 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:37.172238 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 05:09:37.172482 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":140,"threads_started":1}
I20260807 05:09:37.175949 13006 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d7960f76434b7b82f3a0eb85d64bc8" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36283 } }
I20260807 05:09:37.176167 13006 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:37.176222 13006 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4d7960f76434b7b82f3a0eb85d64bc8, State: Initialized, Role: FOLLOWER
I20260807 05:09:37.176604 13006 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d7960f76434b7b82f3a0eb85d64bc8" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36283 } }
I20260807 05:09:37.176733 13006 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:09:37.176793 13006 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:09:37.176887 13006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:37.177811 13006 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d7960f76434b7b82f3a0eb85d64bc8" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36283 } }
I20260807 05:09:37.178134 13006 leader_election.cc:302] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4d7960f76434b7b82f3a0eb85d64bc8; no voters:
I20260807 05:09:37.178426 13006 leader_election.cc:288] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:09:37.178519 13011 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:37.178702 13011 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8 [term 1 LEADER]: Becoming Leader. State: Replica: e4d7960f76434b7b82f3a0eb85d64bc8, State: Running, Role: LEADER
I20260807 05:09:37.178984 13011 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d7960f76434b7b82f3a0eb85d64bc8" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36283 } }
I20260807 05:09:37.179483 13006 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:37.180434 13013 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4d7960f76434b7b82f3a0eb85d64bc8. Latest consensus state: current_term: 1 leader_uuid: "e4d7960f76434b7b82f3a0eb85d64bc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d7960f76434b7b82f3a0eb85d64bc8" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36283 } } }
I20260807 05:09:37.180589 13013 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:37.180676 13012 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4d7960f76434b7b82f3a0eb85d64bc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d7960f76434b7b82f3a0eb85d64bc8" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36283 } } }
I20260807 05:09:37.180778 13012 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:37.181257 13020 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:37.183881 13020 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:37.189246 13020 catalog_manager.cc:1347] Generated new cluster ID: b079b8905f7d49f983b7d8277c9d4985
I20260807 05:09:37.189329 13020 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:37.203107 13020 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:09:37.203428 13020 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:37.216274 13020 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8: Generated new TSK 0
I20260807 05:09:37.216965 13020 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:09:37.219693 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36283
--builtin_ntp_servers=127.8.16.84:46121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 05:09:37.330447 13031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:37.330741 13031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:37.330785 13031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:37.334785 13031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:37.334846 13031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:37.334909 13031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:09:37.339792 13031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:37.340903 13031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:37.342324 13031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:37.351903 13037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:37.351986 13038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:37.352085 13031 server_base.cc:1034] running on GCE node
W20260807 05:09:37.351882 13040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:37.352676 13031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:37.353279 13031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:37.354435 13031 hybrid_clock.cc:648] HybridClock initialized: now 1786079377354420 us; error 48 us; skew 500 ppm
I20260807 05:09:37.356329 13031 webserver.cc:460] Webserver started at http://127.8.16.65:44541/ using document root <none> and password file <none>
I20260807 05:09:37.356916 13031 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:37.356980 13031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:37.357183 13031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:37.359433 13031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "28fa4799dfca4120a5608af8c51d4ea3"
format_stamp: "Formatted at 2026-08-07 05:09:37 on dist-test-slave-49h3"
server_key: "aa02559520da0dc7e65ce197e979702b"
server_key_iv: "7dbe975fb5df80c0b1820b768ca02652"
server_key_version: "encryptionkey@0"
I20260807 05:09:37.359990 13031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "28fa4799dfca4120a5608af8c51d4ea3"
format_stamp: "Formatted at 2026-08-07 05:09:37 on dist-test-slave-49h3"
server_key: "aa02559520da0dc7e65ce197e979702b"
server_key_iv: "7dbe975fb5df80c0b1820b768ca02652"
server_key_version: "encryptionkey@0"
I20260807 05:09:37.363665 13031 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 05:09:37.366130 13046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:37.367316 13031 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:37.367427 13031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "28fa4799dfca4120a5608af8c51d4ea3"
format_stamp: "Formatted at 2026-08-07 05:09:37 on dist-test-slave-49h3"
server_key: "aa02559520da0dc7e65ce197e979702b"
server_key_iv: "7dbe975fb5df80c0b1820b768ca02652"
server_key_version: "encryptionkey@0"
I20260807 05:09:37.367525 13031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:37.371903 13031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:37.372459 13031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:37.372608 13031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:37.373114 13031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:37.373986 13031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:37.374032 13031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:37.374078 13031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:37.374105 13031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:37.382186 13031 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:40491
I20260807 05:09:37.382230 13160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:40491 every 8 connection(s)
I20260807 05:09:37.383020 13031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:09:37.386169 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 13031
I20260807 05:09:37.386298 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:09:37.386590 8257 external_mini_cluster.cc:1428] Setting key 80287fbf0af027edcc76cbbdc3535a01
I20260807 05:09:37.388684 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36283
--builtin_ntp_servers=127.8.16.84:46121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 05:09:37.389065 13009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:37.384637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:60339 (local address 127.8.16.126:36283)
0807 05:09:37.384783 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:37.384787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:37.385853 (+ 1066us) server_negotiation.cc:407] Connection header received
0807 05:09:37.387496 (+ 1643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:37.387498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:37.387533 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:37.387616 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:37.388629 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:37.388631 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:37.388633 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:09:37.388644 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:37.388664 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:09:37.388672 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:37.388732 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:37.388925 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 05:09:37.388964 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 05:09:37.390107 13163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:37.385049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36283 (local address 127.8.16.65:60339)
0807 05:09:37.385715 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:37.385755 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:37.387317 (+ 1562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:37.387738 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:37.387744 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:37.387827 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:37.388338 (+ 511us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:09:37.388340 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:37.388391 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:09:37.388478 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:37.388745 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:37.388754 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:37.388760 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 05:09:37.388961 (+ 201us) client_negotiation.cc:240] Negotiation successful
0807 05:09:37.389200 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":114,"threads_started":1}
I20260807 05:09:37.391263 13161 heartbeater.cc:347] Connected to a master server at 127.8.16.126:36283
I20260807 05:09:37.391471 13161 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:37.391851 13161 heartbeater.cc:511] Master 127.8.16.126:36283 requested a full tablet report, sending...
I20260807 05:09:37.393222 12969 ts_manager.cc:194] Registered new tserver with Master: 28fa4799dfca4120a5608af8c51d4ea3 (127.8.16.65:40491)
W20260807 05:09:37.497095 13164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:37.497364 13164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:37.497421 13164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:37.501370 13164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:37.501423 13164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:37.501516 13164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:09:37.506263 13164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:37.507274 13164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:37.508612 13164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:37.517697 13172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:37.517792 13164 server_base.cc:1034] running on GCE node
W20260807 05:09:37.517747 13169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:37.517699 13170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:37.518308 13164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:37.518893 13164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:37.520047 13164 hybrid_clock.cc:648] HybridClock initialized: now 1786079377520032 us; error 27 us; skew 500 ppm
I20260807 05:09:37.521787 13164 webserver.cc:460] Webserver started at http://127.8.16.66:37201/ using document root <none> and password file <none>
I20260807 05:09:37.522317 13164 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:37.522379 13164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:37.522575 13164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:37.524585 13164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "9720521f437f43caa2ba4b4d6bbb8410"
format_stamp: "Formatted at 2026-08-07 05:09:37 on dist-test-slave-49h3"
server_key: "14badb22ab22494135e47b8d4acab6af"
server_key_iv: "998d04f9bf13acc1b2f028715bbfd486"
server_key_version: "encryptionkey@0"
I20260807 05:09:37.525115 13164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "9720521f437f43caa2ba4b4d6bbb8410"
format_stamp: "Formatted at 2026-08-07 05:09:37 on dist-test-slave-49h3"
server_key: "14badb22ab22494135e47b8d4acab6af"
server_key_iv: "998d04f9bf13acc1b2f028715bbfd486"
server_key_version: "encryptionkey@0"
I20260807 05:09:37.528764 13164 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:09:37.531075 13178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:37.532253 13164 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:37.532351 13164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "9720521f437f43caa2ba4b4d6bbb8410"
format_stamp: "Formatted at 2026-08-07 05:09:37 on dist-test-slave-49h3"
server_key: "14badb22ab22494135e47b8d4acab6af"
server_key_iv: "998d04f9bf13acc1b2f028715bbfd486"
server_key_version: "encryptionkey@0"
I20260807 05:09:37.532457 13164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:37.536948 13164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:37.537576 13164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:37.537724 13164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:37.538233 13164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:37.539083 13164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:37.539126 13164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:37.539165 13164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:37.539193 13164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:37.547355 13164 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:43907
I20260807 05:09:37.547367 13293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:43907 every 8 connection(s)
I20260807 05:09:37.548246 13164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:09:37.553983 13009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:37.550022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:50869 (local address 127.8.16.126:36283)
0807 05:09:37.550151 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:37.550155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:37.551198 (+ 1043us) server_negotiation.cc:407] Connection header received
0807 05:09:37.552512 (+ 1314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:37.552514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:37.552552 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:37.552622 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:37.553624 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:37.553627 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:37.553629 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:09:37.553641 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:37.553661 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:09:37.553677 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:37.553751 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:37.553834 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 05:09:37.553869 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 05:09:37.554839 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:37.550428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36283 (local address 127.8.16.66:50869)
0807 05:09:37.551054 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:37.551094 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:37.552325 (+ 1231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:37.552766 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:37.552774 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:37.552860 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:37.553354 (+ 494us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:09:37.553355 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:37.553393 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:09:37.553485 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:37.553739 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:37.553744 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:37.553747 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 05:09:37.553864 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 05:09:37.554060 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":109,"threads_started":1}
I20260807 05:09:37.555382 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 13164
I20260807 05:09:37.555505 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:09:37.555776 8257 external_mini_cluster.cc:1428] Setting key 3e90f1088108636b1fce51a760e09c85
I20260807 05:09:37.556025 13294 heartbeater.cc:347] Connected to a master server at 127.8.16.126:36283
I20260807 05:09:37.556248 13294 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:37.556685 13294 heartbeater.cc:511] Master 127.8.16.126:36283 requested a full tablet report, sending...
I20260807 05:09:37.557597 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36283
--builtin_ntp_servers=127.8.16.84:46121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 05:09:37.557696 12969 ts_manager.cc:194] Registered new tserver with Master: 9720521f437f43caa2ba4b4d6bbb8410 (127.8.16.66:43907)
W20260807 05:09:37.665792 13297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:37.666088 13297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:37.666136 13297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:37.670194 13297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:37.670251 13297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:37.670321 13297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:09:37.675145 13297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:37.676182 13297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:37.677536 13297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:37.686803 13303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:37.686877 13302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:37.686810 13305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:37.687099 13297 server_base.cc:1034] running on GCE node
I20260807 05:09:37.687495 13297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:37.688076 13297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:37.689229 13297 hybrid_clock.cc:648] HybridClock initialized: now 1786079377689210 us; error 30 us; skew 500 ppm
I20260807 05:09:37.690996 13297 webserver.cc:460] Webserver started at http://127.8.16.67:38185/ using document root <none> and password file <none>
I20260807 05:09:37.691553 13297 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:37.691620 13297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:37.691821 13297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:37.693887 13297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "cd8348c864e54312b943babedf0af430"
format_stamp: "Formatted at 2026-08-07 05:09:37 on dist-test-slave-49h3"
server_key: "fac234f67b0e1a308bb9d77a36456fc4"
server_key_iv: "17d9ebeee0347e3f75e355f74b8d9edc"
server_key_version: "encryptionkey@0"
I20260807 05:09:37.694427 13297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "cd8348c864e54312b943babedf0af430"
format_stamp: "Formatted at 2026-08-07 05:09:37 on dist-test-slave-49h3"
server_key: "fac234f67b0e1a308bb9d77a36456fc4"
server_key_iv: "17d9ebeee0347e3f75e355f74b8d9edc"
server_key_version: "encryptionkey@0"
I20260807 05:09:37.698167 13297 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 05:09:37.700505 13311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:37.701730 13297 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:37.701831 13297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "cd8348c864e54312b943babedf0af430"
format_stamp: "Formatted at 2026-08-07 05:09:37 on dist-test-slave-49h3"
server_key: "fac234f67b0e1a308bb9d77a36456fc4"
server_key_iv: "17d9ebeee0347e3f75e355f74b8d9edc"
server_key_version: "encryptionkey@0"
I20260807 05:09:37.701934 13297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:37.706375 13297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:37.706943 13297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:37.707091 13297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:37.707607 13297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:37.708467 13297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:37.708514 13297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:37.708550 13297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:37.708577 13297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:37.717312 13297 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:43159
I20260807 05:09:37.717339 13425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:43159 every 8 connection(s)
I20260807 05:09:37.718220 13297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:09:37.723708 13009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:37.719874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:40569 (local address 127.8.16.126:36283)
0807 05:09:37.720032 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:37.720037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:37.721069 (+ 1032us) server_negotiation.cc:407] Connection header received
0807 05:09:37.722310 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:37.722313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:37.722352 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:37.722424 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:37.723355 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:37.723358 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:37.723360 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:09:37.723372 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:37.723391 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:09:37.723399 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:37.723469 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:37.723554 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 05:09:37.723589 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 05:09:37.723908 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 13297
I20260807 05:09:37.724038 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:09:37.724324 8257 external_mini_cluster.cc:1428] Setting key d0e81edc5124301aa193fd501c6f45ee
I20260807 05:09:37.724737 13428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:37.720299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36283 (local address 127.8.16.67:40569)
0807 05:09:37.720924 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:37.720966 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:37.722128 (+ 1162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:37.722561 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:37.722567 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:37.722651 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:37.723118 (+ 467us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:09:37.723120 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:37.723158 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:09:37.723222 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:37.723459 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:37.723464 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:37.723467 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 05:09:37.723583 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 05:09:37.723809 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":116,"threads_started":1}
I20260807 05:09:37.725755 13426 heartbeater.cc:347] Connected to a master server at 127.8.16.126:36283
I20260807 05:09:37.725970 13426 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:37.726392 13426 heartbeater.cc:511] Master 127.8.16.126:36283 requested a full tablet report, sending...
I20260807 05:09:37.727326 12969 ts_manager.cc:194] Registered new tserver with Master: cd8348c864e54312b943babedf0af430 (127.8.16.67:43159)
I20260807 05:09:37.738277 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:09:37.741611 13009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:37.740152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52532 (local address 127.8.16.126:36283)
0807 05:09:37.740280 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:37.740284 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:37.740429 (+ 145us) server_negotiation.cc:407] Connection header received
0807 05:09:37.740530 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:37.740532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:37.740564 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:37.740650 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:37.741286 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:37.741289 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:37.741291 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:09:37.741301 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:37.741321 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:09:37.741329 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:37.741393 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:37.741441 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 05:09:37.741501 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 05:09:37.744259 12969 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52532:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:09:37.747299 12969 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:09:37.758628 13439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:37.755611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60234 (local address 127.8.16.66:43907)
0807 05:09:37.756603 (+ 992us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:37.756613 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:37.756821 (+ 208us) server_negotiation.cc:407] Connection header received
0807 05:09:37.756879 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:37.756885 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:37.757030 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:37.757148 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:37.758040 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:37.758047 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:37.758059 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:09:37.758071 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:37.758122 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:09:37.758160 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:37.758262 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:37.758402 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 05:09:37.758472 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":903,"thread_start_us":106,"threads_started":1}
I20260807 05:09:37.758617 13441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:37.755414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:43907 (local address 127.0.0.1:60234)
0807 05:09:37.756626 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:37.756638 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:37.756731 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:37.757432 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:37.757435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:37.757482 (+ 47us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:37.757919 (+ 437us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:09:37.757920 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:37.757926 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:09:37.757938 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:37.758231 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:37.758241 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:37.758247 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 05:09:37.758355 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 05:09:37.758411 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1153,"thread_start_us":103,"threads_started":1}
I20260807 05:09:37.758636 13438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:37.755350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:43159 (local address 127.0.0.1:42428)
0807 05:09:37.756096 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:37.756143 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:37.756244 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:37.757050 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:37.757057 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:37.757135 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:37.757737 (+ 602us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:09:37.757739 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:37.757758 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:09:37.757818 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:37.758262 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:37.758265 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:37.758269 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 05:09:37.758339 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 05:09:37.758421 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":2,"thread_start_us":96,"threads_started":1}
I20260807 05:09:37.758965 13442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:37.756171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42428 (local address 127.8.16.67:43159)
0807 05:09:37.756584 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:37.756596 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:37.756620 (+ 24us) server_negotiation.cc:407] Connection header received
0807 05:09:37.756676 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:37.756682 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:37.756899 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:37.757042 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:37.758075 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:37.758089 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:37.758097 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:09:37.758104 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:37.758135 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:09:37.758150 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:37.758512 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:37.758559 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 05:09:37.758624 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":108,"threads_started":1}
I20260807 05:09:37.759397 13437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:37.755627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:40491 (local address 127.0.0.1:39904)
0807 05:09:37.756725 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:37.756736 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:37.756803 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:37.758015 (+ 1212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:37.758017 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:37.758030 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:37.758800 (+ 770us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:09:37.758801 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:37.758811 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:09:37.758838 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:37.759187 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:37.759190 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:37.759194 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 05:09:37.759256 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 05:09:37.759289 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1049,"thread_start_us":105,"threads_started":1}
I20260807 05:09:37.759891 13443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:37.756144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39904 (local address 127.8.16.65:40491)
0807 05:09:37.756491 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:37.756502 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:37.756832 (+ 330us) server_negotiation.cc:407] Connection header received
0807 05:09:37.756890 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:37.756895 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:37.757046 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:37.757159 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:37.758965 (+ 1806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:37.758970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:37.758980 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:09:37.758992 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:37.759033 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:09:37.759065 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:37.759149 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:37.759652 (+ 503us) server_negotiation.cc:299] Negotiation successful
0807 05:09:37.759732 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":133,"threads_started":1}
I20260807 05:09:37.761700 13228 tablet_service.cc:1467] Processing CreateTablet for tablet 9bdcf64f92a447c89d938726074c698d (DEFAULT_TABLE table=test-table [id=677fbac443d742ef9ee1e2fa140b7ebd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:37.761844 13360 tablet_service.cc:1467] Processing CreateTablet for tablet 9bdcf64f92a447c89d938726074c698d (DEFAULT_TABLE table=test-table [id=677fbac443d742ef9ee1e2fa140b7ebd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:37.762329 13094 tablet_service.cc:1467] Processing CreateTablet for tablet 9bdcf64f92a447c89d938726074c698d (DEFAULT_TABLE table=test-table [id=677fbac443d742ef9ee1e2fa140b7ebd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:37.762748 13228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bdcf64f92a447c89d938726074c698d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:37.762851 13360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bdcf64f92a447c89d938726074c698d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:37.763228 13094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bdcf64f92a447c89d938726074c698d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:37.768692 13444 tablet_bootstrap.cc:492] T 9bdcf64f92a447c89d938726074c698d P 9720521f437f43caa2ba4b4d6bbb8410: Bootstrap starting.
I20260807 05:09:37.770229 13445 tablet_bootstrap.cc:492] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430: Bootstrap starting.
I20260807 05:09:37.770267 13446 tablet_bootstrap.cc:492] T 9bdcf64f92a447c89d938726074c698d P 28fa4799dfca4120a5608af8c51d4ea3: Bootstrap starting.
I20260807 05:09:37.771147 13444 tablet_bootstrap.cc:654] T 9bdcf64f92a447c89d938726074c698d P 9720521f437f43caa2ba4b4d6bbb8410: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:37.772053 13444 log.cc:824] T 9bdcf64f92a447c89d938726074c698d P 9720521f437f43caa2ba4b4d6bbb8410: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:37.772316 13445 tablet_bootstrap.cc:654] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:37.772316 13446 tablet_bootstrap.cc:654] T 9bdcf64f92a447c89d938726074c698d P 28fa4799dfca4120a5608af8c51d4ea3: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:37.773123 13445 log.cc:824] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:37.773124 13446 log.cc:824] T 9bdcf64f92a447c89d938726074c698d P 28fa4799dfca4120a5608af8c51d4ea3: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:37.774031 13444 tablet_bootstrap.cc:492] T 9bdcf64f92a447c89d938726074c698d P 9720521f437f43caa2ba4b4d6bbb8410: No bootstrap required, opened a new log
I20260807 05:09:37.774227 13444 ts_tablet_manager.cc:1397] T 9bdcf64f92a447c89d938726074c698d P 9720521f437f43caa2ba4b4d6bbb8410: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 05:09:37.774955 13446 tablet_bootstrap.cc:492] T 9bdcf64f92a447c89d938726074c698d P 28fa4799dfca4120a5608af8c51d4ea3: No bootstrap required, opened a new log
I20260807 05:09:37.774955 13445 tablet_bootstrap.cc:492] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430: No bootstrap required, opened a new log
I20260807 05:09:37.775128 13446 ts_tablet_manager.cc:1397] T 9bdcf64f92a447c89d938726074c698d P 28fa4799dfca4120a5608af8c51d4ea3: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 05:09:37.775128 13445 ts_tablet_manager.cc:1397] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 05:09:37.777858 13444 raft_consensus.cc:348] T 9bdcf64f92a447c89d938726074c698d P 9720521f437f43caa2ba4b4d6bbb8410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8348c864e54312b943babedf0af430" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 43159 } } peers { permanent_uuid: "9720521f437f43caa2ba4b4d6bbb8410" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 43907 } } peers { permanent_uuid: "28fa4799dfca4120a5608af8c51d4ea3" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 40491 } }
I20260807 05:09:37.778069 13444 raft_consensus.cc:374] T 9bdcf64f92a447c89d938726074c698d P 9720521f437f43caa2ba4b4d6bbb8410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:37.778118 13444 raft_consensus.cc:729] T 9bdcf64f92a447c89d938726074c698d P 9720521f437f43caa2ba4b4d6bbb8410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9720521f437f43caa2ba4b4d6bbb8410, State: Initialized, Role: FOLLOWER
I20260807 05:09:37.778513 13444 consensus_queue.cc:260] T 9bdcf64f92a447c89d938726074c698d P 9720521f437f43caa2ba4b4d6bbb8410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8348c864e54312b943babedf0af430" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 43159 } } peers { permanent_uuid: "9720521f437f43caa2ba4b4d6bbb8410" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 43907 } } peers { permanent_uuid: "28fa4799dfca4120a5608af8c51d4ea3" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 40491 } }
I20260807 05:09:37.779129 13294 heartbeater.cc:503] Master 127.8.16.126:36283 was elected leader, sending a full tablet report...
I20260807 05:09:37.779300 13446 raft_consensus.cc:348] T 9bdcf64f92a447c89d938726074c698d P 28fa4799dfca4120a5608af8c51d4ea3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8348c864e54312b943babedf0af430" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 43159 } } peers { permanent_uuid: "9720521f437f43caa2ba4b4d6bbb8410" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 43907 } } peers { permanent_uuid: "28fa4799dfca4120a5608af8c51d4ea3" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 40491 } }
I20260807 05:09:37.779300 13445 raft_consensus.cc:348] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8348c864e54312b943babedf0af430" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 43159 } } peers { permanent_uuid: "9720521f437f43caa2ba4b4d6bbb8410" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 43907 } } peers { permanent_uuid: "28fa4799dfca4120a5608af8c51d4ea3" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 40491 } }
I20260807 05:09:37.779510 13444 ts_tablet_manager.cc:1428] T 9bdcf64f92a447c89d938726074c698d P 9720521f437f43caa2ba4b4d6bbb8410: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 05:09:37.779528 13446 raft_consensus.cc:374] T 9bdcf64f92a447c89d938726074c698d P 28fa4799dfca4120a5608af8c51d4ea3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:37.779528 13445 raft_consensus.cc:374] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:37.779584 13446 raft_consensus.cc:729] T 9bdcf64f92a447c89d938726074c698d P 28fa4799dfca4120a5608af8c51d4ea3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28fa4799dfca4120a5608af8c51d4ea3, State: Initialized, Role: FOLLOWER
I20260807 05:09:37.779577 13445 raft_consensus.cc:729] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd8348c864e54312b943babedf0af430, State: Initialized, Role: FOLLOWER
I20260807 05:09:37.780030 13445 consensus_queue.cc:260] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8348c864e54312b943babedf0af430" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 43159 } } peers { permanent_uuid: "9720521f437f43caa2ba4b4d6bbb8410" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 43907 } } peers { permanent_uuid: "28fa4799dfca4120a5608af8c51d4ea3" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 40491 } }
I20260807 05:09:37.780073 13446 consensus_queue.cc:260] T 9bdcf64f92a447c89d938726074c698d P 28fa4799dfca4120a5608af8c51d4ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8348c864e54312b943babedf0af430" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 43159 } } peers { permanent_uuid: "9720521f437f43caa2ba4b4d6bbb8410" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 43907 } } peers { permanent_uuid: "28fa4799dfca4120a5608af8c51d4ea3" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 40491 } }
I20260807 05:09:37.780975 13426 heartbeater.cc:503] Master 127.8.16.126:36283 was elected leader, sending a full tablet report...
I20260807 05:09:37.781015 13445 ts_tablet_manager.cc:1428] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260807 05:09:37.781015 13446 ts_tablet_manager.cc:1428] T 9bdcf64f92a447c89d938726074c698d P 28fa4799dfca4120a5608af8c51d4ea3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 05:09:37.781842 13161 heartbeater.cc:503] Master 127.8.16.126:36283 was elected leader, sending a full tablet report...
I20260807 05:09:37.790171 13452 raft_consensus.cc:482] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:09:37.790334 13452 raft_consensus.cc:504] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8348c864e54312b943babedf0af430" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 43159 } } peers { permanent_uuid: "9720521f437f43caa2ba4b4d6bbb8410" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 43907 } } peers { permanent_uuid: "28fa4799dfca4120a5608af8c51d4ea3" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 40491 } }
I20260807 05:09:37.791333 13452 leader_election.cc:288] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9720521f437f43caa2ba4b4d6bbb8410 (127.8.16.66:43907), 28fa4799dfca4120a5608af8c51d4ea3 (127.8.16.65:40491)
I20260807 05:09:37.793061 13439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:37.791617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:47579 (local address 127.8.16.66:43907)
0807 05:09:37.791864 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:37.791869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:37.791883 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:09:37.791924 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:37.791926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:37.791960 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:37.792048 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:37.792693 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:37.792697 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:37.792699 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:09:37.792710 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:37.792732 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:09:37.792740 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:37.792821 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:37.792902 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 05:09:37.792939 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168}
I20260807 05:09:37.793061 13428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:37.791523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:43907 (local address 127.8.16.67:47579)
0807 05:09:37.791739 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:37.791755 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:37.791829 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:37.792065 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:37.792068 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:37.792083 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:37.792555 (+ 472us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:09:37.792557 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:37.792570 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:09:37.792596 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:37.792823 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:37.792827 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:37.792831 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 05:09:37.792905 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 05:09:37.792939 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260807 05:09:37.793154 13443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:37.791633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:56897 (local address 127.8.16.65:40491)
0807 05:09:37.791749 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:37.791753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:37.792040 (+ 287us) server_negotiation.cc:407] Connection header received
0807 05:09:37.792205 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:37.792208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:37.792246 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:37.792319 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:37.792828 (+ 509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:37.792831 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:37.792834 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:09:37.792845 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:37.792864 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:09:37.792871 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:37.792940 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:37.793005 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 05:09:37.793039 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 05:09:37.793195 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:37.791565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:40491 (local address 127.8.16.67:56897)
0807 05:09:37.791968 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:37.791980 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:37.792092 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:37.792347 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:37.792349 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:37.792363 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:37.792701 (+ 338us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:09:37.792703 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:37.792710 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:09:37.792730 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:37.792935 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:37.792937 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:37.792940 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 05:09:37.793007 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 05:09:37.793039 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":128,"threads_started":1}
I20260807 05:09:37.793792 13248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bdcf64f92a447c89d938726074c698d" candidate_uuid: "cd8348c864e54312b943babedf0af430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9720521f437f43caa2ba4b4d6bbb8410" is_pre_election: true
I20260807 05:09:37.794046 13248 raft_consensus.cc:2454] T 9bdcf64f92a447c89d938726074c698d P 9720521f437f43caa2ba4b4d6bbb8410 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd8348c864e54312b943babedf0af430 in term 0.
I20260807 05:09:37.794075 13114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bdcf64f92a447c89d938726074c698d" candidate_uuid: "cd8348c864e54312b943babedf0af430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28fa4799dfca4120a5608af8c51d4ea3" is_pre_election: true
I20260807 05:09:37.794325 13114 raft_consensus.cc:2454] T 9bdcf64f92a447c89d938726074c698d P 28fa4799dfca4120a5608af8c51d4ea3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd8348c864e54312b943babedf0af430 in term 0.
I20260807 05:09:37.794505 13312 leader_election.cc:302] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9720521f437f43caa2ba4b4d6bbb8410, cd8348c864e54312b943babedf0af430; no voters:
I20260807 05:09:37.794749 13452 raft_consensus.cc:2781] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:09:37.794840 13452 raft_consensus.cc:482] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:09:37.794891 13452 raft_consensus.cc:3037] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:37.795877 13452 raft_consensus.cc:504] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8348c864e54312b943babedf0af430" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 43159 } } peers { permanent_uuid: "9720521f437f43caa2ba4b4d6bbb8410" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 43907 } } peers { permanent_uuid: "28fa4799dfca4120a5608af8c51d4ea3" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 40491 } }
I20260807 05:09:37.796169 13452 leader_election.cc:288] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430 [CANDIDATE]: Term 1 election: Requested vote from peers 9720521f437f43caa2ba4b4d6bbb8410 (127.8.16.66:43907), 28fa4799dfca4120a5608af8c51d4ea3 (127.8.16.65:40491)
I20260807 05:09:37.796478 13248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bdcf64f92a447c89d938726074c698d" candidate_uuid: "cd8348c864e54312b943babedf0af430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9720521f437f43caa2ba4b4d6bbb8410"
I20260807 05:09:37.796587 13248 raft_consensus.cc:3037] T 9bdcf64f92a447c89d938726074c698d P 9720521f437f43caa2ba4b4d6bbb8410 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:37.797616 13248 raft_consensus.cc:2454] T 9bdcf64f92a447c89d938726074c698d P 9720521f437f43caa2ba4b4d6bbb8410 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd8348c864e54312b943babedf0af430 in term 1.
I20260807 05:09:37.797861 13114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bdcf64f92a447c89d938726074c698d" candidate_uuid: "cd8348c864e54312b943babedf0af430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28fa4799dfca4120a5608af8c51d4ea3"
I20260807 05:09:37.797894 13312 leader_election.cc:302] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9720521f437f43caa2ba4b4d6bbb8410, cd8348c864e54312b943babedf0af430; no voters:
I20260807 05:09:37.797966 13114 raft_consensus.cc:3037] T 9bdcf64f92a447c89d938726074c698d P 28fa4799dfca4120a5608af8c51d4ea3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:37.798043 13452 raft_consensus.cc:2781] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:37.798309 13452 raft_consensus.cc:686] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430 [term 1 LEADER]: Becoming Leader. State: Replica: cd8348c864e54312b943babedf0af430, State: Running, Role: LEADER
I20260807 05:09:37.798641 13452 consensus_queue.cc:237] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8348c864e54312b943babedf0af430" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 43159 } } peers { permanent_uuid: "9720521f437f43caa2ba4b4d6bbb8410" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 43907 } } peers { permanent_uuid: "28fa4799dfca4120a5608af8c51d4ea3" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 40491 } }
W20260807 05:09:37.799108 13295 tablet.cc:2369] T 9bdcf64f92a447c89d938726074c698d P 9720521f437f43caa2ba4b4d6bbb8410: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:09:37.799211 13114 raft_consensus.cc:2454] T 9bdcf64f92a447c89d938726074c698d P 28fa4799dfca4120a5608af8c51d4ea3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd8348c864e54312b943babedf0af430 in term 1.
I20260807 05:09:37.801986 12967 catalog_manager.cc:5495] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd8348c864e54312b943babedf0af430 (127.8.16.67). New cstate: current_term: 1 leader_uuid: "cd8348c864e54312b943babedf0af430" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8348c864e54312b943babedf0af430" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 43159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9720521f437f43caa2ba4b4d6bbb8410" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 43907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28fa4799dfca4120a5608af8c51d4ea3" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 40491 } health_report { overall_health: UNKNOWN } } }
I20260807 05:09:37.821707 13442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:37.820391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42440 (local address 127.8.16.67:43159)
0807 05:09:37.820500 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:37.820503 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:37.820517 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:09:37.820651 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:37.820653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:37.820711 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:37.820781 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:37.821380 (+ 599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:37.821383 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:37.821385 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:09:37.821391 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:37.821404 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:09:37.821411 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:37.821498 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:37.821572 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 05:09:37.821604 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 05:09:37.827241 13114 raft_consensus.cc:1261] T 9bdcf64f92a447c89d938726074c698d P 28fa4799dfca4120a5608af8c51d4ea3 [term 1 FOLLOWER]: Refusing update from remote peer cd8348c864e54312b943babedf0af430: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:37.827241 13248 raft_consensus.cc:1261] T 9bdcf64f92a447c89d938726074c698d P 9720521f437f43caa2ba4b4d6bbb8410 [term 1 FOLLOWER]: Refusing update from remote peer cd8348c864e54312b943babedf0af430: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:37.827946 13454 consensus_queue.cc:1035] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28fa4799dfca4120a5608af8c51d4ea3" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 40491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:37.828122 13452 consensus_queue.cc:1035] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9720521f437f43caa2ba4b4d6bbb8410" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 43907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:37.835484 13457 mvcc.cc:205] Tried to move back new op lower bound from 7315781131573714944 to 7315781131466387456. Current Snapshot: MvccSnapshot[applied={T|T < 7315781131573714944}]
I20260807 05:09:37.836663 13459 mvcc.cc:205] Tried to move back new op lower bound from 7315781131573714944 to 7315781131466387456. Current Snapshot: MvccSnapshot[applied={T|T < 7315781131573714944}]
I20260807 05:09:37.836526 13458 mvcc.cc:205] Tried to move back new op lower bound from 7315781131573714944 to 7315781131466387456. Current Snapshot: MvccSnapshot[applied={T|T < 7315781131573714944}]
I20260807 05:09:37.843878 12968 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:37.844061 12968 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:37.846431 12968 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e4d7960f76434b7b82f3a0eb85d64bc8: Sending DeleteTablet for 3 replicas of tablet 9bdcf64f92a447c89d938726074c698d
I20260807 05:09:37.847139 13360 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bdcf64f92a447c89d938726074c698d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:37 UTC) from {username='slave'} at 127.0.0.1:42428
I20260807 05:09:37.847151 13094 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bdcf64f92a447c89d938726074c698d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:37 UTC) from {username='slave'} at 127.0.0.1:39904
I20260807 05:09:37.847265 13228 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bdcf64f92a447c89d938726074c698d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:37 UTC) from {username='slave'} at 127.0.0.1:60234
I20260807 05:09:37.847640 13468 tablet_replica.cc:330] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430: stopping tablet replica
I20260807 05:09:37.847935 13468 raft_consensus.cc:2229] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:09:37.848290 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 13031
I20260807 05:09:37.848362 13468 raft_consensus.cc:2258] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:37.850235 13468 ts_tablet_manager.cc:1905] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:37.853005 13468 ts_tablet_manager.cc:1918] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:09:37.853106 13468 log.cc:1196] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/9bdcf64f92a447c89d938726074c698d
I20260807 05:09:37.853407 13468 ts_tablet_manager.cc:1939] T 9bdcf64f92a447c89d938726074c698d P cd8348c864e54312b943babedf0af430: Deleting consensus metadata
I20260807 05:09:37.854633 12955 catalog_manager.cc:4841] TS cd8348c864e54312b943babedf0af430 (127.8.16.67:43159): tablet 9bdcf64f92a447c89d938726074c698d (table test-table [id=677fbac443d742ef9ee1e2fa140b7ebd]) successfully deleted
I20260807 05:09:37.854876 13469 tablet_replica.cc:330] T 9bdcf64f92a447c89d938726074c698d P 9720521f437f43caa2ba4b4d6bbb8410: stopping tablet replica
I20260807 05:09:37.855213 13469 raft_consensus.cc:2229] T 9bdcf64f92a447c89d938726074c698d P 9720521f437f43caa2ba4b4d6bbb8410 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:37.855526 13469 raft_consensus.cc:2258] T 9bdcf64f92a447c89d938726074c698d P 9720521f437f43caa2ba4b4d6bbb8410 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 05:09:37.856576 12954 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.16.65:40491 (error 108)
I20260807 05:09:37.857054 13469 ts_tablet_manager.cc:1905] T 9bdcf64f92a447c89d938726074c698d P 9720521f437f43caa2ba4b4d6bbb8410: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:37.857546 13437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:37.857257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:40491 (local address 127.0.0.1:39916)
0807 05:09:37.857378 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:37.857441 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.65:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 05:09:37.857630 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 13164
W20260807 05:09:37.858327 12954 catalog_manager.cc:4584] TS 28fa4799dfca4120a5608af8c51d4ea3 (127.8.16.65:40491): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9bdcf64f92a447c89d938726074c698d: Network error: Client connection negotiation failed: client connection to 127.8.16.65:40491: connect: Connection refused (error 111)
I20260807 05:09:37.866146 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 13297
I20260807 05:09:37.867439 13437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:37.866253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:43907 (local address 127.0.0.1:60248)
0807 05:09:37.867291 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:37.867349 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.66:43907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":943}
I20260807 05:09:37.871968 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 12938
2026-08-07T05:09:37Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (898 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 05:09:37.893507 8257 mini_oidc.cc:152] Starting JWKS server
I20260807 05:09:37.895263 8257 webserver.cc:460] Webserver started at https://127.0.0.1:34631/ using document root <none> and password file <none>
I20260807 05:09:37.895433 8257 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 05:09:37.895700 8257 webserver.cc:460] Webserver started at http://127.0.0.1:45457/ using document root <none> and password file <none>
2026-08-07T05:09:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:37Z Disabled control of system clock
I20260807 05:09:37.905014 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:35497
--webserver_interface=127.8.16.126
--webserver_port=0
--builtin_ntp_servers=127.8.16.84:32865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:35497
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34631/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079354071073-8257-0/testchainca.cert with env {}
W20260807 05:09:38.016165 13476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:38.016422 13476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:38.016461 13476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:38.020444 13476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:38.020501 13476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:38.020516 13476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:38.020537 13476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:38.020561 13476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 05:09:38.020591 13476 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 05:09:38.020606 13476 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34631/jwks.json
I20260807 05:09:38.026005 13476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:32865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:35497
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:35497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34631/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079354071073-8257-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:38.027102 13476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:38.028100 13476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:38.036180 13481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:38.036186 13482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:38.036338 13476 server_base.cc:1034] running on GCE node
W20260807 05:09:38.036228 13484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:38.037020 13476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:38.038112 13476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:38.039273 13476 hybrid_clock.cc:648] HybridClock initialized: now 1786079378039260 us; error 31 us; skew 500 ppm
I20260807 05:09:38.041113 13476 webserver.cc:460] Webserver started at http://127.8.16.126:33747/ using document root <none> and password file <none>
I20260807 05:09:38.041682 13476 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:38.041747 13476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:38.041916 13476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:38.043895 13476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "8aad6ee1d9c94199a1b9d8392b142dbb"
format_stamp: "Formatted at 2026-08-07 05:09:38 on dist-test-slave-49h3"
server_key: "1f520b47760df8ef19efee673cdfbbf7"
server_key_iv: "b07518084c0f2dad4be19ff6d33a1c77"
server_key_version: "encryptionkey@0"
I20260807 05:09:38.044409 13476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "8aad6ee1d9c94199a1b9d8392b142dbb"
format_stamp: "Formatted at 2026-08-07 05:09:38 on dist-test-slave-49h3"
server_key: "1f520b47760df8ef19efee673cdfbbf7"
server_key_iv: "b07518084c0f2dad4be19ff6d33a1c77"
server_key_version: "encryptionkey@0"
I20260807 05:09:38.047994 13476 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260807 05:09:38.050395 13490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:38.051559 13476 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:38.051666 13476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "8aad6ee1d9c94199a1b9d8392b142dbb"
format_stamp: "Formatted at 2026-08-07 05:09:38 on dist-test-slave-49h3"
server_key: "1f520b47760df8ef19efee673cdfbbf7"
server_key_iv: "b07518084c0f2dad4be19ff6d33a1c77"
server_key_version: "encryptionkey@0"
I20260807 05:09:38.051752 13476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:38.068158 13476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:38.068846 13476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:38.069001 13476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:38.075672 13476 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:35497
I20260807 05:09:38.075688 13544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:35497 every 8 connection(s)
I20260807 05:09:38.076618 13476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:38.079310 13545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:38.080790 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 13476
I20260807 05:09:38.080888 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:09:38.081154 8257 external_mini_cluster.cc:1428] Setting key 3578216d5c27d2c533c5c44d16f591dd
I20260807 05:09:38.085955 13545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8aad6ee1d9c94199a1b9d8392b142dbb: Bootstrap starting.
I20260807 05:09:38.088445 13545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8aad6ee1d9c94199a1b9d8392b142dbb: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:38.089162 13545 log.cc:824] T 00000000000000000000000000000000 P 8aad6ee1d9c94199a1b9d8392b142dbb: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:38.091159 13548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:38.082592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46644 (local address 127.8.16.126:35497)
0807 05:09:38.083274 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:38.083296 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:38.083339 (+ 43us) server_negotiation.cc:407] Connection header received
0807 05:09:38.084079 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:38.084129 (+ 50us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:38.084614 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:38.084983 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:38.085400 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:38.086827 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:38.087675 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:38.088342 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:38.088501 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:38.089195 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:38.089201 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:38.089215 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:09:38.089696 (+ 481us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:38.089750 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:09:38.089779 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:38.089849 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:38.090018 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 05:09:38.090269 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":102,"threads_started":1}
I20260807 05:09:38.091313 13545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8aad6ee1d9c94199a1b9d8392b142dbb: No bootstrap required, opened a new log
I20260807 05:09:38.093712 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:09:38.094841 13545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8aad6ee1d9c94199a1b9d8392b142dbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aad6ee1d9c94199a1b9d8392b142dbb" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35497 } }
I20260807 05:09:38.095052 13545 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8aad6ee1d9c94199a1b9d8392b142dbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:38.095106 13545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8aad6ee1d9c94199a1b9d8392b142dbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aad6ee1d9c94199a1b9d8392b142dbb, State: Initialized, Role: FOLLOWER
I20260807 05:09:38.095530 13545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8aad6ee1d9c94199a1b9d8392b142dbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aad6ee1d9c94199a1b9d8392b142dbb" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35497 } }
I20260807 05:09:38.095651 13545 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8aad6ee1d9c94199a1b9d8392b142dbb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:09:38.095696 13545 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8aad6ee1d9c94199a1b9d8392b142dbb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:09:38.095760 13545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8aad6ee1d9c94199a1b9d8392b142dbb [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:38.096599 13545 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8aad6ee1d9c94199a1b9d8392b142dbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aad6ee1d9c94199a1b9d8392b142dbb" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35497 } }
I20260807 05:09:38.096876 13545 leader_election.cc:302] T 00000000000000000000000000000000 P 8aad6ee1d9c94199a1b9d8392b142dbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8aad6ee1d9c94199a1b9d8392b142dbb; no voters:
I20260807 05:09:38.097119 13545 leader_election.cc:288] T 00000000000000000000000000000000 P 8aad6ee1d9c94199a1b9d8392b142dbb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:09:38.097282 13551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8aad6ee1d9c94199a1b9d8392b142dbb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:38.097517 13551 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8aad6ee1d9c94199a1b9d8392b142dbb [term 1 LEADER]: Becoming Leader. State: Replica: 8aad6ee1d9c94199a1b9d8392b142dbb, State: Running, Role: LEADER
I20260807 05:09:38.097811 13551 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8aad6ee1d9c94199a1b9d8392b142dbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aad6ee1d9c94199a1b9d8392b142dbb" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35497 } }
I20260807 05:09:38.098338 13545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8aad6ee1d9c94199a1b9d8392b142dbb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:38.099483 13552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aad6ee1d9c94199a1b9d8392b142dbb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8aad6ee1d9c94199a1b9d8392b142dbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aad6ee1d9c94199a1b9d8392b142dbb" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35497 } } }
I20260807 05:09:38.099494 13553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aad6ee1d9c94199a1b9d8392b142dbb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8aad6ee1d9c94199a1b9d8392b142dbb. Latest consensus state: current_term: 1 leader_uuid: "8aad6ee1d9c94199a1b9d8392b142dbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aad6ee1d9c94199a1b9d8392b142dbb" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35497 } } }
I20260807 05:09:38.099606 13552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aad6ee1d9c94199a1b9d8392b142dbb [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:38.099611 13553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aad6ee1d9c94199a1b9d8392b142dbb [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:38.099936 13560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:38.102484 13560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:38.107684 13560 catalog_manager.cc:1347] Generated new cluster ID: c8cd024a3fc44a99b42a2339b0215e80
I20260807 05:09:38.107752 13560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:38.120078 13560 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:09:38.121312 13560 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:38.139143 13560 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8aad6ee1d9c94199a1b9d8392b142dbb: Generated new TSK 0
I20260807 05:09:38.140046 13560 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 05:09:38.171738 13576 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 05:09:38.171880 13576 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 05:09:38.172519 13548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:38.154941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46650 (local address 127.8.16.126:35497)
0807 05:09:38.155069 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:38.155073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:38.155282 (+ 209us) server_negotiation.cc:407] Connection header received
0807 05:09:38.155471 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:38.155474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:38.155531 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:38.155643 (+ 112us) server_negotiation.cc:226] Negotiated authn=JWT
0807 05:09:38.155839 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:38.156701 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:38.157549 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:38.157905 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:38.157995 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:38.158273 (+ 278us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 05:09:38.172070 (+ 13797us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34631/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 05:09:38.172366 (+ 296us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46650: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34631/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 05:09:38.172609 13548 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46650: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34631/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 05:09:38.173774 13575 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:38.154830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:35497 (local address 127.0.0.1:46650)
0807 05:09:38.155206 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:38.155221 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:38.155329 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:38.155643 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:38.155645 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:38.155651 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0807 05:09:38.155753 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:38.155757 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:38.156801 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:38.156804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:38.157427 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:38.157434 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:38.158006 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:38.158009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:38.158071 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:38.158112 (+ 41us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 05:09:38.172296 (+ 14184us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34631/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 05:09:38.172502 (+ 206us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.16.126:35497: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34631/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":116,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.8.16.126:35497: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34631/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 05:09:38.174494 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 13476
2026-08-07T05:09:38Z chronyd exiting
I20260807 05:09:38.212087 8257 test_util.cc:175] -----------------------------------------------
I20260807 05:09:38.212183 8257 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079354071073-8257-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (320 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 05:09:38.213299 8257 mini_oidc.cc:152] Starting JWKS server
I20260807 05:09:38.214201 8257 webserver.cc:460] Webserver started at https://127.0.0.1:40573/ using document root <none> and password file <none>
I20260807 05:09:38.214286 8257 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 05:09:38.214493 8257 webserver.cc:460] Webserver started at http://127.0.0.1:41779/ using document root <none> and password file <none>
2026-08-07T05:09:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:38Z Disabled control of system clock
I20260807 05:09:38.223624 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:40945
--webserver_interface=127.8.16.126
--webserver_port=0
--builtin_ntp_servers=127.8.16.84:37315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:40945
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40573/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079354071073-8257-0/testchainca.cert with env {}
W20260807 05:09:38.330955 13583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:38.331210 13583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:38.331249 13583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:38.335147 13583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:38.335198 13583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:38.335211 13583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:38.335227 13583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:38.335245 13583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 05:09:38.335259 13583 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 05:09:38.335273 13583 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40573/jwks.json
I20260807 05:09:38.340551 13583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:37315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:40945
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:40945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40573/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079354071073-8257-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:38.341545 13583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:38.342482 13583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:38.350584 13591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:38.350634 13588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:38.350590 13589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:38.350844 13583 server_base.cc:1034] running on GCE node
I20260807 05:09:38.351388 13583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:38.352406 13583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:38.353519 13583 hybrid_clock.cc:648] HybridClock initialized: now 1786079378353501 us; error 32 us; skew 500 ppm
I20260807 05:09:38.355201 13583 webserver.cc:460] Webserver started at http://127.8.16.126:33697/ using document root <none> and password file <none>
I20260807 05:09:38.355741 13583 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:38.355808 13583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:38.356002 13583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:38.358011 13583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "1f5dc4c3954447599781458260574739"
format_stamp: "Formatted at 2026-08-07 05:09:38 on dist-test-slave-49h3"
server_key: "6d0b03f70e411815bc41cc43f5c441d0"
server_key_iv: "814f26d8c251a8eda9bf204f9124ee6f"
server_key_version: "encryptionkey@0"
I20260807 05:09:38.358527 13583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "1f5dc4c3954447599781458260574739"
format_stamp: "Formatted at 2026-08-07 05:09:38 on dist-test-slave-49h3"
server_key: "6d0b03f70e411815bc41cc43f5c441d0"
server_key_iv: "814f26d8c251a8eda9bf204f9124ee6f"
server_key_version: "encryptionkey@0"
I20260807 05:09:38.361943 13583 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:09:38.364127 13597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:38.365302 13583 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:38.365424 13583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "1f5dc4c3954447599781458260574739"
format_stamp: "Formatted at 2026-08-07 05:09:38 on dist-test-slave-49h3"
server_key: "6d0b03f70e411815bc41cc43f5c441d0"
server_key_iv: "814f26d8c251a8eda9bf204f9124ee6f"
server_key_version: "encryptionkey@0"
I20260807 05:09:38.365562 13583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:38.380136 13583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:38.380800 13583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:38.380985 13583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:38.387494 13583 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:40945
I20260807 05:09:38.387508 13652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:40945 every 8 connection(s)
I20260807 05:09:38.388482 13583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:38.391554 13653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:38.397075 13653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f5dc4c3954447599781458260574739: Bootstrap starting.
I20260807 05:09:38.398667 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 13583
I20260807 05:09:38.398787 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:09:38.399050 8257 external_mini_cluster.cc:1428] Setting key 472129dd246b323f966be669dfee6bfa
I20260807 05:09:38.399688 13653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f5dc4c3954447599781458260574739: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:38.400509 13653 log.cc:824] T 00000000000000000000000000000000 P 1f5dc4c3954447599781458260574739: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:38.402806 13653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f5dc4c3954447599781458260574739: No bootstrap required, opened a new log
I20260807 05:09:38.407017 13653 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f5dc4c3954447599781458260574739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f5dc4c3954447599781458260574739" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 40945 } }
I20260807 05:09:38.407303 13653 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f5dc4c3954447599781458260574739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:38.407377 13653 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f5dc4c3954447599781458260574739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f5dc4c3954447599781458260574739, State: Initialized, Role: FOLLOWER
I20260807 05:09:38.407853 13653 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f5dc4c3954447599781458260574739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f5dc4c3954447599781458260574739" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 40945 } }
I20260807 05:09:38.408012 13653 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f5dc4c3954447599781458260574739 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:09:38.408067 13653 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f5dc4c3954447599781458260574739 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:09:38.408147 13653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f5dc4c3954447599781458260574739 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:38.409119 13656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:38.400218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42916 (local address 127.8.16.126:40945)
0807 05:09:38.400941 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:38.400957 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:38.400993 (+ 36us) server_negotiation.cc:407] Connection header received
0807 05:09:38.401774 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:38.401797 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:38.402201 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:38.402545 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:38.402989 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:38.404443 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:38.405333 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:38.406129 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:38.406299 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:38.407077 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:38.407084 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:38.407099 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:09:38.407581 (+ 482us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:38.407653 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:09:38.407691 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:38.407781 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:38.407888 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 05:09:38.408161 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":121,"threads_started":1}
I20260807 05:09:38.409267 13653 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f5dc4c3954447599781458260574739 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f5dc4c3954447599781458260574739" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 40945 } }
I20260807 05:09:38.409638 13653 leader_election.cc:302] T 00000000000000000000000000000000 P 1f5dc4c3954447599781458260574739 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f5dc4c3954447599781458260574739; no voters:
I20260807 05:09:38.409981 13653 leader_election.cc:288] T 00000000000000000000000000000000 P 1f5dc4c3954447599781458260574739 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:09:38.410084 13658 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f5dc4c3954447599781458260574739 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:38.410311 13658 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f5dc4c3954447599781458260574739 [term 1 LEADER]: Becoming Leader. State: Replica: 1f5dc4c3954447599781458260574739, State: Running, Role: LEADER
I20260807 05:09:38.410597 13658 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f5dc4c3954447599781458260574739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f5dc4c3954447599781458260574739" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 40945 } }
I20260807 05:09:38.411146 13653 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f5dc4c3954447599781458260574739 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:38.412209 13660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f5dc4c3954447599781458260574739 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f5dc4c3954447599781458260574739. Latest consensus state: current_term: 1 leader_uuid: "1f5dc4c3954447599781458260574739" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f5dc4c3954447599781458260574739" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 40945 } } }
I20260807 05:09:38.412253 13659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f5dc4c3954447599781458260574739 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f5dc4c3954447599781458260574739" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f5dc4c3954447599781458260574739" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 40945 } } }
I20260807 05:09:38.412330 13660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f5dc4c3954447599781458260574739 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:38.412361 13659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f5dc4c3954447599781458260574739 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:38.412669 13668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:38.412691 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:09:38.416132 13668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:38.421530 13668 catalog_manager.cc:1347] Generated new cluster ID: 54a0174e6e6542f49d74992c53144d40
I20260807 05:09:38.421586 13668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:38.426028 13668 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:09:38.426841 13668 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:38.437233 13668 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f5dc4c3954447599781458260574739: Generated new TSK 0
I20260807 05:09:38.438063 13668 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 05:09:38.470011 13685 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 05:09:38.470093 13685 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260807 05:09:38.470867 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:38.454937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:40945 (local address 127.0.0.1:42924)
0807 05:09:38.455232 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:38.455244 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:38.455345 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:38.455622 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:38.455624 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:38.455630 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0807 05:09:38.455723 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:38.455726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:38.456724 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:38.456731 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:38.457355 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:38.457362 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:38.457930 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:38.457932 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:38.457991 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:38.458013 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 05:09:38.470575 (+ 12562us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40573/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 05:09:38.470724 (+ 149us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.16.126:40945: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40573/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":107,"threads_started":1}
I20260807 05:09:38.471161 13656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:38.455048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42924 (local address 127.8.16.126:40945)
0807 05:09:38.455173 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:38.455178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:38.455315 (+ 137us) server_negotiation.cc:407] Connection header received
0807 05:09:38.455450 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:38.455453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:38.455511 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:38.455619 (+ 108us) server_negotiation.cc:226] Negotiated authn=JWT
0807 05:09:38.455801 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:38.456627 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:38.457474 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:38.457830 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:38.457929 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:38.458166 (+ 237us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 05:09:38.470392 (+ 12226us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40573/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 05:09:38.470650 (+ 258us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42924: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40573/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":41}
W20260807 05:09:38.471235 13656 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42924: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40573/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260807 05:09:38.471658 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 13583
2026-08-07T05:09:38Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (297 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 05:09:38.510814 8257 mini_oidc.cc:152] Starting JWKS server
I20260807 05:09:38.511672 8257 webserver.cc:460] Webserver started at https://127.0.0.1:43255/ using document root <none> and password file <none>
I20260807 05:09:38.511776 8257 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 05:09:38.512028 8257 webserver.cc:460] Webserver started at http://127.0.0.1:43897/ using document root <none> and password file <none>
2026-08-07T05:09:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:38Z Disabled control of system clock
I20260807 05:09:38.520555 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:43937
--webserver_interface=127.8.16.126
--webserver_port=0
--builtin_ntp_servers=127.8.16.84:38613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:43937
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43255/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 05:09:38.626010 13691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:38.626291 13691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:38.626344 13691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:38.630251 13691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:38.630307 13691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:38.630327 13691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:38.630348 13691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:38.630367 13691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 05:09:38.630393 13691 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 05:09:38.630419 13691 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43255/jwks.json
I20260807 05:09:38.635039 13691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:38613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:43937
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:43937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43255/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:38.636111 13691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:38.637467 13691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:38.645434 13697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:38.645465 13696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:38.645480 13699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:38.645568 13691 server_base.cc:1034] running on GCE node
I20260807 05:09:38.646318 13691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:38.647414 13691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:38.648576 13691 hybrid_clock.cc:648] HybridClock initialized: now 1786079378648558 us; error 29 us; skew 500 ppm
I20260807 05:09:38.650410 13691 webserver.cc:460] Webserver started at http://127.8.16.126:45673/ using document root <none> and password file <none>
I20260807 05:09:38.650964 13691 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:38.651031 13691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:38.651242 13691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:38.653321 13691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "d56051909acc4a88951f45bd65382989"
format_stamp: "Formatted at 2026-08-07 05:09:38 on dist-test-slave-49h3"
server_key: "43df1ff27a916f6c8350150c11b52cab"
server_key_iv: "a7b871d124ba1f8fe78b86371e21d0cd"
server_key_version: "encryptionkey@0"
I20260807 05:09:38.653928 13691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "d56051909acc4a88951f45bd65382989"
format_stamp: "Formatted at 2026-08-07 05:09:38 on dist-test-slave-49h3"
server_key: "43df1ff27a916f6c8350150c11b52cab"
server_key_iv: "a7b871d124ba1f8fe78b86371e21d0cd"
server_key_version: "encryptionkey@0"
I20260807 05:09:38.657513 13691 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 05:09:38.659847 13706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:38.661024 13691 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 05:09:38.661164 13691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "d56051909acc4a88951f45bd65382989"
format_stamp: "Formatted at 2026-08-07 05:09:38 on dist-test-slave-49h3"
server_key: "43df1ff27a916f6c8350150c11b52cab"
server_key_iv: "a7b871d124ba1f8fe78b86371e21d0cd"
server_key_version: "encryptionkey@0"
I20260807 05:09:38.661274 13691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:38.670934 13691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:38.671535 13691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:38.671690 13691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:38.678095 13691 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:43937
I20260807 05:09:38.678112 13758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:43937 every 8 connection(s)
I20260807 05:09:38.679003 13691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:38.682125 13759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:38.685730 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 13691
I20260807 05:09:38.685863 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:09:38.686196 8257 external_mini_cluster.cc:1428] Setting key 69f535d850bb4546a97a3f263b9f0681
I20260807 05:09:38.687956 13759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d56051909acc4a88951f45bd65382989: Bootstrap starting.
I20260807 05:09:38.690133 13759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d56051909acc4a88951f45bd65382989: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:38.690828 13759 log.cc:824] T 00000000000000000000000000000000 P d56051909acc4a88951f45bd65382989: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:38.692533 13759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d56051909acc4a88951f45bd65382989: No bootstrap required, opened a new log
I20260807 05:09:38.694957 13762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:38.687597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60130 (local address 127.8.16.126:43937)
0807 05:09:38.688251 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:38.688267 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:38.688304 (+ 37us) server_negotiation.cc:407] Connection header received
0807 05:09:38.688917 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:38.688933 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:38.689318 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:38.689670 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:38.690094 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:38.691102 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:38.691964 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:38.692421 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:38.692561 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:38.693351 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:38.693356 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:38.693366 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:09:38.693775 (+ 409us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:38.693827 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:09:38.693847 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:38.693929 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:38.694057 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 05:09:38.694295 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":136,"threads_started":1}
I20260807 05:09:38.696005 13759 raft_consensus.cc:348] T 00000000000000000000000000000000 P d56051909acc4a88951f45bd65382989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d56051909acc4a88951f45bd65382989" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43937 } }
I20260807 05:09:38.696223 13759 raft_consensus.cc:374] T 00000000000000000000000000000000 P d56051909acc4a88951f45bd65382989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:38.696280 13759 raft_consensus.cc:729] T 00000000000000000000000000000000 P d56051909acc4a88951f45bd65382989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d56051909acc4a88951f45bd65382989, State: Initialized, Role: FOLLOWER
I20260807 05:09:38.696702 13759 consensus_queue.cc:260] T 00000000000000000000000000000000 P d56051909acc4a88951f45bd65382989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d56051909acc4a88951f45bd65382989" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43937 } }
I20260807 05:09:38.696839 13759 raft_consensus.cc:388] T 00000000000000000000000000000000 P d56051909acc4a88951f45bd65382989 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:09:38.696906 13759 raft_consensus.cc:482] T 00000000000000000000000000000000 P d56051909acc4a88951f45bd65382989 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:09:38.697001 13759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d56051909acc4a88951f45bd65382989 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:38.697353 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:09:38.697896 13759 raft_consensus.cc:504] T 00000000000000000000000000000000 P d56051909acc4a88951f45bd65382989 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d56051909acc4a88951f45bd65382989" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43937 } }
I20260807 05:09:38.698202 13759 leader_election.cc:302] T 00000000000000000000000000000000 P d56051909acc4a88951f45bd65382989 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d56051909acc4a88951f45bd65382989; no voters:
I20260807 05:09:38.698514 13759 leader_election.cc:288] T 00000000000000000000000000000000 P d56051909acc4a88951f45bd65382989 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:09:38.698575 13764 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d56051909acc4a88951f45bd65382989 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:38.698802 13764 raft_consensus.cc:686] T 00000000000000000000000000000000 P d56051909acc4a88951f45bd65382989 [term 1 LEADER]: Becoming Leader. State: Replica: d56051909acc4a88951f45bd65382989, State: Running, Role: LEADER
I20260807 05:09:38.699115 13764 consensus_queue.cc:237] T 00000000000000000000000000000000 P d56051909acc4a88951f45bd65382989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d56051909acc4a88951f45bd65382989" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43937 } }
I20260807 05:09:38.699705 13759 sys_catalog.cc:564] T 00000000000000000000000000000000 P d56051909acc4a88951f45bd65382989 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:38.700696 13765 sys_catalog.cc:455] T 00000000000000000000000000000000 P d56051909acc4a88951f45bd65382989 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d56051909acc4a88951f45bd65382989" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d56051909acc4a88951f45bd65382989" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43937 } } }
I20260807 05:09:38.700825 13765 sys_catalog.cc:458] T 00000000000000000000000000000000 P d56051909acc4a88951f45bd65382989 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:38.701069 13766 sys_catalog.cc:455] T 00000000000000000000000000000000 P d56051909acc4a88951f45bd65382989 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d56051909acc4a88951f45bd65382989. Latest consensus state: current_term: 1 leader_uuid: "d56051909acc4a88951f45bd65382989" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d56051909acc4a88951f45bd65382989" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43937 } } }
I20260807 05:09:38.701165 13766 sys_catalog.cc:458] T 00000000000000000000000000000000 P d56051909acc4a88951f45bd65382989 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:38.701273 13773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:38.703977 13773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:38.709355 13773 catalog_manager.cc:1347] Generated new cluster ID: 33b7dabbdf5f4fd6805d3d0feb0f3476
I20260807 05:09:38.709424 13773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:38.733057 13773 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:09:38.734035 13773 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:38.742342 13773 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d56051909acc4a88951f45bd65382989: Generated new TSK 0
I20260807 05:09:38.743027 13773 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:09:38.760488 13762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:38.755626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60144 (local address 127.8.16.126:43937)
0807 05:09:38.755762 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:38.755767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:38.755936 (+ 169us) server_negotiation.cc:407] Connection header received
0807 05:09:38.756096 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:38.756098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:38.756144 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:38.756231 (+ 87us) server_negotiation.cc:226] Negotiated authn=JWT
0807 05:09:38.756470 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:38.756998 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:38.758229 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:38.758451 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:38.758517 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:38.758823 (+ 306us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 05:09:38.760112 (+ 1289us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43255/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 05:09:38.760342 (+ 230us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60144: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43255/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 05:09:38.760560 13762 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60144: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43255/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 05:09:38.762365 13788 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:38.755542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:43937 (local address 127.0.0.1:60144)
0807 05:09:38.755852 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:38.755867 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:38.755985 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:38.756246 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:38.756249 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:38.756257 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0807 05:09:38.756373 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:38.756379 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:38.757093 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:38.757097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:38.758118 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:38.758128 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:38.758551 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:38.758554 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:38.758643 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:38.758681 (+ 38us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 05:09:38.760274 (+ 1593us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43255/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 05:09:38.762183 (+ 1909us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.16.126:43937: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43255/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":1702,"thread_start_us":90,"threads_started":1}
I20260807 05:09:38.763060 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 13691
E20260807 05:09:38.773739 13789 webserver.cc:533] Webserver: sslize failed:
E20260807 05:09:38.773820 13789 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-07T05:09:38Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (292 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T05:09:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:38Z Disabled control of system clock
I20260807 05:09:38.812650 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:42097
--webserver_interface=127.8.16.126
--webserver_port=0
--builtin_ntp_servers=127.8.16.84:40679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:42097
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079354071073-8257-0/testchainca.cert with env {}
W20260807 05:09:38.918325 13793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:38.918586 13793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:38.918643 13793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:38.922602 13793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:38.922669 13793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:38.922691 13793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:38.922712 13793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:38.922740 13793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 05:09:38.922770 13793 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 05:09:38.922791 13793 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 05:09:38.928189 13793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:40679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:42097
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:42097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079354071073-8257-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:38.929230 13793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:38.930207 13793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:38.938293 13799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:38.938301 13800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:38.938278 13802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:38.938593 13793 server_base.cc:1034] running on GCE node
I20260807 05:09:38.939215 13793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:38.940254 13793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:38.941421 13793 hybrid_clock.cc:648] HybridClock initialized: now 1786079378941406 us; error 25 us; skew 500 ppm
I20260807 05:09:38.943189 13793 webserver.cc:460] Webserver started at http://127.8.16.126:43623/ using document root <none> and password file <none>
I20260807 05:09:38.943737 13793 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:38.943800 13793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:38.943995 13793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:38.946192 13793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "1597bca625c84d18a7672db2c6c8ab5f"
format_stamp: "Formatted at 2026-08-07 05:09:38 on dist-test-slave-49h3"
server_key: "275d51e22edfb53427b5abfa9ac91582"
server_key_iv: "1850f5af7bbada824f66e4bc6c8004df"
server_key_version: "encryptionkey@0"
I20260807 05:09:38.946800 13793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "1597bca625c84d18a7672db2c6c8ab5f"
format_stamp: "Formatted at 2026-08-07 05:09:38 on dist-test-slave-49h3"
server_key: "275d51e22edfb53427b5abfa9ac91582"
server_key_iv: "1850f5af7bbada824f66e4bc6c8004df"
server_key_version: "encryptionkey@0"
I20260807 05:09:38.950471 13793 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:09:38.952769 13809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:38.953959 13793 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:38.954075 13793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "1597bca625c84d18a7672db2c6c8ab5f"
format_stamp: "Formatted at 2026-08-07 05:09:38 on dist-test-slave-49h3"
server_key: "275d51e22edfb53427b5abfa9ac91582"
server_key_iv: "1850f5af7bbada824f66e4bc6c8004df"
server_key_version: "encryptionkey@0"
I20260807 05:09:38.954176 13793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:38.968065 13793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:38.968736 13793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:38.968894 13793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:38.975504 13793 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:42097
I20260807 05:09:38.975521 13864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:42097 every 8 connection(s)
I20260807 05:09:38.976440 13793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:38.977689 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 13793
I20260807 05:09:38.977809 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:09:38.978072 8257 external_mini_cluster.cc:1428] Setting key 0d777bc804f59f1e0d9f81d0b0e33fa8
I20260807 05:09:38.979825 13865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:38.985911 13865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1597bca625c84d18a7672db2c6c8ab5f: Bootstrap starting.
I20260807 05:09:38.987135 13868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:38.979386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35504 (local address 127.8.16.126:42097)
0807 05:09:38.979992 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:38.980007 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:38.980042 (+ 35us) server_negotiation.cc:407] Connection header received
0807 05:09:38.980667 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:38.980682 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:38.981057 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:38.981331 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:38.981712 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:38.982678 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:38.983826 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:38.984294 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:38.984405 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:38.985340 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:38.985346 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:38.985359 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:09:38.985803 (+ 444us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:38.985862 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:09:38.985883 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:38.985957 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:38.986181 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 05:09:38.986445 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":119,"threads_started":1}
I20260807 05:09:38.988142 13865 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1597bca625c84d18a7672db2c6c8ab5f: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:38.988832 13865 log.cc:824] T 00000000000000000000000000000000 P 1597bca625c84d18a7672db2c6c8ab5f: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:38.989765 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:09:38.990863 13865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1597bca625c84d18a7672db2c6c8ab5f: No bootstrap required, opened a new log
I20260807 05:09:38.994110 13865 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1597bca625c84d18a7672db2c6c8ab5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1597bca625c84d18a7672db2c6c8ab5f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42097 } }
I20260807 05:09:38.994325 13865 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1597bca625c84d18a7672db2c6c8ab5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:38.994375 13865 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1597bca625c84d18a7672db2c6c8ab5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1597bca625c84d18a7672db2c6c8ab5f, State: Initialized, Role: FOLLOWER
I20260807 05:09:38.994757 13865 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1597bca625c84d18a7672db2c6c8ab5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1597bca625c84d18a7672db2c6c8ab5f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42097 } }
I20260807 05:09:38.994885 13865 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1597bca625c84d18a7672db2c6c8ab5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:09:38.994947 13865 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1597bca625c84d18a7672db2c6c8ab5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:09:38.995035 13865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1597bca625c84d18a7672db2c6c8ab5f [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:38.995865 13865 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1597bca625c84d18a7672db2c6c8ab5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1597bca625c84d18a7672db2c6c8ab5f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42097 } }
I20260807 05:09:38.996160 13865 leader_election.cc:302] T 00000000000000000000000000000000 P 1597bca625c84d18a7672db2c6c8ab5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1597bca625c84d18a7672db2c6c8ab5f; no voters:
I20260807 05:09:38.996438 13865 leader_election.cc:288] T 00000000000000000000000000000000 P 1597bca625c84d18a7672db2c6c8ab5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:09:38.996558 13870 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1597bca625c84d18a7672db2c6c8ab5f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:38.996765 13870 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1597bca625c84d18a7672db2c6c8ab5f [term 1 LEADER]: Becoming Leader. State: Replica: 1597bca625c84d18a7672db2c6c8ab5f, State: Running, Role: LEADER
I20260807 05:09:38.997066 13870 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1597bca625c84d18a7672db2c6c8ab5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1597bca625c84d18a7672db2c6c8ab5f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42097 } }
I20260807 05:09:38.997668 13865 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1597bca625c84d18a7672db2c6c8ab5f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:38.998143 13872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1597bca625c84d18a7672db2c6c8ab5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1597bca625c84d18a7672db2c6c8ab5f. Latest consensus state: current_term: 1 leader_uuid: "1597bca625c84d18a7672db2c6c8ab5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1597bca625c84d18a7672db2c6c8ab5f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42097 } } }
I20260807 05:09:38.998270 13872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1597bca625c84d18a7672db2c6c8ab5f [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:38.998622 13875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:38.998898 13871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1597bca625c84d18a7672db2c6c8ab5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1597bca625c84d18a7672db2c6c8ab5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1597bca625c84d18a7672db2c6c8ab5f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42097 } } }
I20260807 05:09:38.999002 13871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1597bca625c84d18a7672db2c6c8ab5f [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:39.002000 13875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:39.007732 13875 catalog_manager.cc:1347] Generated new cluster ID: fb116baf57ab49249a5d77d73e76cb09
I20260807 05:09:39.007804 13875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:39.020416 13875 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:09:39.021282 13875 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:39.027824 13875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1597bca625c84d18a7672db2c6c8ab5f: Generated new TSK 0
I20260807 05:09:39.028453 13875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:09:39.084057 13868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:39.042290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35512 (local address 127.8.16.126:42097)
0807 05:09:39.042403 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:39.042408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:39.042599 (+ 191us) server_negotiation.cc:407] Connection header received
0807 05:09:39.042778 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:39.042780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:39.042827 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:39.042914 (+ 87us) server_negotiation.cc:226] Negotiated authn=JWT
0807 05:09:39.043142 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:39.043672 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:39.044504 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:39.044712 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:39.044796 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:39.045094 (+ 298us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 05:09:39.083615 (+ 38521us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 05:09:39.083909 (+ 294us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35512: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":38}
W20260807 05:09:39.084149 13868 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35512: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 05:09:39.084221 13894 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:39.042234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42097 (local address 127.0.0.1:35512)
0807 05:09:39.042518 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:39.042530 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:39.042637 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:39.042926 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:39.042929 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:39.042937 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0807 05:09:39.043045 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:39.043051 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:39.043782 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:39.043785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:39.044393 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:39.044400 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:39.044820 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:39.044823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:39.044916 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:39.044939 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 05:09:39.083843 (+ 38904us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 05:09:39.084037 (+ 194us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.16.126:42097: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":115,"threads_started":1}
I20260807 05:09:39.084641 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 13793
2026-08-07T05:09:39Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (300 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (121 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (129 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:09:39 dist-test-slave-49h3 krb5kdc[13906](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:09:39 dist-test-slave-49h3 krb5kdc[13906](info): set up 2 sockets
Aug 07 05:09:39 dist-test-slave-49h3 krb5kdc[13906](info): commencing operation
krb5kdc: starting...
W20260807 05:09:40.322211 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.967s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:09:40 dist-test-slave-49h3 krb5kdc[13906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:09:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:09:40.438059 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:37493
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:44095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:37493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:40.544597 13919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:40.544852 13919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:40.544891 13919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:40.548839 13919 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:40.548898 13919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:40.548918 13919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:40.548931 13919 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:40.548952 13919 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:40.554270 13919 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:44095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:37493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:37493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:40.555491 13919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:40.556701 13919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:40.566675 13925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:40.566704 13927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:40.566871 13919 server_base.cc:1034] running on GCE node
W20260807 05:09:40.566704 13924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:40.567559 13919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:40.568570 13919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:40.569720 13919 hybrid_clock.cc:648] HybridClock initialized: now 1786079380569695 us; error 41 us; skew 500 ppm
Aug 07 05:09:40 dist-test-slave-49h3 krb5kdc[13906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079380, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:40.572245 13919 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:09:40.573069 13919 webserver.cc:460] Webserver started at http://127.8.16.126:43779/ using document root <none> and password file <none>
I20260807 05:09:40.573613 13919 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:40.573675 13919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:40.573845 13919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:40.575793 13919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "00eebe520f004cfdb207a135999a41ab"
format_stamp: "Formatted at 2026-08-07 05:09:40 on dist-test-slave-49h3"
server_key: "1b6f54a86d145313733b49e67d1fcc21"
server_key_iv: "dc45d272ed95a4306b09893684aa15c2"
server_key_version: "encryptionkey@0"
I20260807 05:09:40.576295 13919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "00eebe520f004cfdb207a135999a41ab"
format_stamp: "Formatted at 2026-08-07 05:09:40 on dist-test-slave-49h3"
server_key: "1b6f54a86d145313733b49e67d1fcc21"
server_key_iv: "dc45d272ed95a4306b09893684aa15c2"
server_key_version: "encryptionkey@0"
I20260807 05:09:40.579761 13919 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260807 05:09:40.582006 13935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:40.583128 13919 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:40.583240 13919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "00eebe520f004cfdb207a135999a41ab"
format_stamp: "Formatted at 2026-08-07 05:09:40 on dist-test-slave-49h3"
server_key: "1b6f54a86d145313733b49e67d1fcc21"
server_key_iv: "dc45d272ed95a4306b09893684aa15c2"
server_key_version: "encryptionkey@0"
I20260807 05:09:40.583333 13919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:40.598981 13919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:40.601744 13919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:40.601919 13919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:40.608508 13919 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:37493
I20260807 05:09:40.608537 13990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:37493 every 8 connection(s)
I20260807 05:09:40.609434 13919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:40.612121 13991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:40.613538 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 13919
I20260807 05:09:40.613660 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:09:40.613943 8257 external_mini_cluster.cc:1428] Setting key 31457e82473e7939591163cc5735e60b
I20260807 05:09:40.618469 13991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00eebe520f004cfdb207a135999a41ab: Bootstrap starting.
I20260807 05:09:40.620986 13991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00eebe520f004cfdb207a135999a41ab: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:40.621762 13991 log.cc:824] T 00000000000000000000000000000000 P 00eebe520f004cfdb207a135999a41ab: Log is configured to *not* fsync() on all Append() calls
Aug 07 05:09:40 dist-test-slave-49h3 krb5kdc[13906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:40.623625 13991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00eebe520f004cfdb207a135999a41ab: No bootstrap required, opened a new log
I20260807 05:09:40.627918 13991 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00eebe520f004cfdb207a135999a41ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00eebe520f004cfdb207a135999a41ab" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37493 } }
I20260807 05:09:40.628180 13991 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00eebe520f004cfdb207a135999a41ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:40.628185 13994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:40.615327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57056 (local address 127.8.16.126:37493)
0807 05:09:40.615758 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:40.615777 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:40.615821 (+ 44us) server_negotiation.cc:407] Connection header received
0807 05:09:40.616628 (+ 807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:40.616650 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:40.617062 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:40.617418 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:40.618102 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:40.619490 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:40.620297 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:40.620969 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:40.621128 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:40.623436 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:40.623442 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:40.623456 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:40.623485 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:40.625295 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:40.625716 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:40.625722 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:40.625728 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:40.625881 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:40.626112 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:40.626116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:40.626117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:40.626604 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:40.626807 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:40.626968 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 05:09:40.627258 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":108,"threads_started":1}
I20260807 05:09:40.628249 13991 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00eebe520f004cfdb207a135999a41ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00eebe520f004cfdb207a135999a41ab, State: Initialized, Role: FOLLOWER
I20260807 05:09:40.628654 13991 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00eebe520f004cfdb207a135999a41ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00eebe520f004cfdb207a135999a41ab" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37493 } }
I20260807 05:09:40.628773 13991 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00eebe520f004cfdb207a135999a41ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:09:40.628831 13991 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00eebe520f004cfdb207a135999a41ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:09:40.628924 13991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00eebe520f004cfdb207a135999a41ab [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:40.629989 13991 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00eebe520f004cfdb207a135999a41ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00eebe520f004cfdb207a135999a41ab" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37493 } }
I20260807 05:09:40.630336 13991 leader_election.cc:302] T 00000000000000000000000000000000 P 00eebe520f004cfdb207a135999a41ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00eebe520f004cfdb207a135999a41ab; no voters:
I20260807 05:09:40.630630 13991 leader_election.cc:288] T 00000000000000000000000000000000 P 00eebe520f004cfdb207a135999a41ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:09:40.630738 13996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00eebe520f004cfdb207a135999a41ab [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:40.630941 13996 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00eebe520f004cfdb207a135999a41ab [term 1 LEADER]: Becoming Leader. State: Replica: 00eebe520f004cfdb207a135999a41ab, State: Running, Role: LEADER
I20260807 05:09:40.631206 13996 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00eebe520f004cfdb207a135999a41ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00eebe520f004cfdb207a135999a41ab" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37493 } }
I20260807 05:09:40.631747 13991 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00eebe520f004cfdb207a135999a41ab [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:40.632804 13998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00eebe520f004cfdb207a135999a41ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00eebe520f004cfdb207a135999a41ab. Latest consensus state: current_term: 1 leader_uuid: "00eebe520f004cfdb207a135999a41ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00eebe520f004cfdb207a135999a41ab" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37493 } } }
I20260807 05:09:40.632828 13997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00eebe520f004cfdb207a135999a41ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00eebe520f004cfdb207a135999a41ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00eebe520f004cfdb207a135999a41ab" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37493 } } }
I20260807 05:09:40.632939 13998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00eebe520f004cfdb207a135999a41ab [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:40.632939 13997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00eebe520f004cfdb207a135999a41ab [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:40.633287 14005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:40.636330 14005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:40.641412 14005 catalog_manager.cc:1347] Generated new cluster ID: 91c4c67e44d040a59d5dcb258c086176
I20260807 05:09:40.641489 14005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:40.651029 14005 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:09:40.652232 14005 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:40.656198 14005 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00eebe520f004cfdb207a135999a41ab: Generated new TSK 0
I20260807 05:09:40.656960 14005 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
I20260807 05:09:40.684419 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37493
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:44095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:40.793130 14018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:40.793392 14018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:40.793439 14018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:40.797398 14018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:40.797482 14018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:40.797564 14018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:09:40.802817 14018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:44095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:40.803876 14018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:40.804891 14018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:40.814268 14023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:40.814375 14018 server_base.cc:1034] running on GCE node
W20260807 05:09:40.814262 14026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:40.814261 14024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:40.814934 14018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:40.815541 14018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:40.816689 14018 hybrid_clock.cc:648] HybridClock initialized: now 1786079380816665 us; error 42 us; skew 500 ppm
Aug 07 05:09:40 dist-test-slave-49h3 krb5kdc[13906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079380, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:40.819360 14018 init.cc:377] Logged in from keytab as kudu/127.8.16.65@KRBTEST.COM (short username kudu)
I20260807 05:09:40.820226 14018 webserver.cc:460] Webserver started at http://127.8.16.65:45797/ using document root <none> and password file <none>
I20260807 05:09:40.820721 14018 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:40.820778 14018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:40.820938 14018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:40.822923 14018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "3a72a653743a407a8e915ed509524b1a"
format_stamp: "Formatted at 2026-08-07 05:09:40 on dist-test-slave-49h3"
server_key: "251fdcab0ff51920329bf8cabcdc7278"
server_key_iv: "13604e142c6f6a6c7443c2d252dd497c"
server_key_version: "encryptionkey@0"
I20260807 05:09:40.823426 14018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "3a72a653743a407a8e915ed509524b1a"
format_stamp: "Formatted at 2026-08-07 05:09:40 on dist-test-slave-49h3"
server_key: "251fdcab0ff51920329bf8cabcdc7278"
server_key_iv: "13604e142c6f6a6c7443c2d252dd497c"
server_key_version: "encryptionkey@0"
I20260807 05:09:40.826948 14018 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:09:40.829149 14033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:40.830353 14018 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:40.830443 14018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "3a72a653743a407a8e915ed509524b1a"
format_stamp: "Formatted at 2026-08-07 05:09:40 on dist-test-slave-49h3"
server_key: "251fdcab0ff51920329bf8cabcdc7278"
server_key_iv: "13604e142c6f6a6c7443c2d252dd497c"
server_key_version: "encryptionkey@0"
I20260807 05:09:40.830523 14018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:40.848201 14018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:40.851394 14018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:40.851588 14018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:40.852164 14018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:40.852979 14018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:40.853025 14018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:40.853068 14018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:40.853099 14018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:40.861210 14018 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:44311
I20260807 05:09:40.861248 14150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:44311 every 8 connection(s)
I20260807 05:09:40.862107 14018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:09:40.870850 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 14018
I20260807 05:09:40.870975 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
Aug 07 05:09:40 dist-test-slave-49h3 krb5kdc[13906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079380, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:40.871277 8257 external_mini_cluster.cc:1428] Setting key 0f35f68125df330a18b1d2e096f65852
I20260807 05:09:40.875304 13994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:40.863760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:35499 (local address 127.8.16.126:37493)
0807 05:09:40.863880 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:40.863883 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:40.864657 (+ 774us) server_negotiation.cc:407] Connection header received
0807 05:09:40.866144 (+ 1487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:40.866148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:40.866222 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:40.866294 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:40.867426 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:40.867949 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:40.869051 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:40.869288 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:40.869376 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:40.871777 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:40.871781 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:40.871784 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:40.871813 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:40.873212 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:40.873772 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:40.873775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:40.873776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:40.873848 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:40.874186 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:40.874188 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:40.874189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:40.874345 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:40.874415 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:40.874997 (+ 582us) server_negotiation.cc:299] Negotiation successful
0807 05:09:40.875124 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 05:09:40.876022 14153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:40.864079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:37493 (local address 127.8.16.65:35499)
0807 05:09:40.864512 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:40.864554 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:40.865932 (+ 1378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:40.866462 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:40.866469 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:40.866820 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:40.867280 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:40.867292 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:40.868066 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:40.868069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:40.868931 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:40.868940 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:40.869422 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:40.869427 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:40.869609 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:40.870124 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:40.870139 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:40.871612 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:40.873321 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:40.873327 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:40.873341 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:40.873687 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:40.873921 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:40.873924 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:40.873926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:40.874101 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:40.874428 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:40.874434 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:40.874730 (+ 296us) client_negotiation.cc:769] Sending connection context
0807 05:09:40.874929 (+ 199us) client_negotiation.cc:240] Negotiation successful
0807 05:09:40.875194 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":112,"threads_started":1}
I20260807 05:09:40.877117 14151 heartbeater.cc:347] Connected to a master server at 127.8.16.126:37493
I20260807 05:09:40.877381 14151 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:40.877983 14151 heartbeater.cc:511] Master 127.8.16.126:37493 requested a full tablet report, sending...
I20260807 05:09:40.879575 13953 ts_manager.cc:194] Registered new tserver with Master: 3a72a653743a407a8e915ed509524b1a (127.8.16.65:44311)
I20260807 05:09:40.880703 13953 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:35499
WARNING: no policy specified for kudu/127.8.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
I20260807 05:09:40.897768 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37493
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:44095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:41.005209 14158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:41.005522 14158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:41.005584 14158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:41.009608 14158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:41.009686 14158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:41.009771 14158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:09:41.015096 14158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:44095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:41.016216 14158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:41.017303 14158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:41.026678 14164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:41.026672 14163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:41.026669 14166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:41.026878 14158 server_base.cc:1034] running on GCE node
I20260807 05:09:41.027364 14158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:41.027987 14158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:41.029150 14158 hybrid_clock.cc:648] HybridClock initialized: now 1786079381029133 us; error 29 us; skew 500 ppm
Aug 07 05:09:41 dist-test-slave-49h3 krb5kdc[13906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079381, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:41.036811 14158 init.cc:377] Logged in from keytab as kudu/127.8.16.66@KRBTEST.COM (short username kudu)
I20260807 05:09:41.037815 14158 webserver.cc:460] Webserver started at http://127.8.16.66:45543/ using document root <none> and password file <none>
I20260807 05:09:41.038369 14158 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:41.038439 14158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:41.038643 14158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:41.040701 14158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "7a0540eb25fc44d792255eab52714611"
format_stamp: "Formatted at 2026-08-07 05:09:41 on dist-test-slave-49h3"
server_key: "097eac622f16176f64ba7eb8a7c7aafb"
server_key_iv: "f1ff6cf4a60799b3eacea92434277a56"
server_key_version: "encryptionkey@0"
I20260807 05:09:41.041242 14158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "7a0540eb25fc44d792255eab52714611"
format_stamp: "Formatted at 2026-08-07 05:09:41 on dist-test-slave-49h3"
server_key: "097eac622f16176f64ba7eb8a7c7aafb"
server_key_iv: "f1ff6cf4a60799b3eacea92434277a56"
server_key_version: "encryptionkey@0"
I20260807 05:09:41.044847 14158 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:09:41.047453 14175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:41.048676 14158 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 05:09:41.048794 14158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "7a0540eb25fc44d792255eab52714611"
format_stamp: "Formatted at 2026-08-07 05:09:41 on dist-test-slave-49h3"
server_key: "097eac622f16176f64ba7eb8a7c7aafb"
server_key_iv: "f1ff6cf4a60799b3eacea92434277a56"
server_key_version: "encryptionkey@0"
I20260807 05:09:41.048905 14158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:41.067106 14158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:41.070060 14158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:41.070264 14158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:41.070828 14158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:41.071627 14158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:41.071676 14158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:41.071727 14158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:41.071756 14158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:41.080078 14158 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:46297
I20260807 05:09:41.080107 14292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:46297 every 8 connection(s)
I20260807 05:09:41.080960 14158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:09:41.084093 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 14158
I20260807 05:09:41.084206 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:09:41.084466 8257 external_mini_cluster.cc:1428] Setting key 23548648053c3d454e9054928ded80d1
Aug 07 05:09:41 dist-test-slave-49h3 krb5kdc[13906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079381, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.16.67@KRBTEST.COM; defaulting to no policy
I20260807 05:09:41.094857 13994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:41.082711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:45079 (local address 127.8.16.126:37493)
0807 05:09:41.082839 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:41.082843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:41.083672 (+ 829us) server_negotiation.cc:407] Connection header received
0807 05:09:41.085059 (+ 1387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:41.085062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:41.085118 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:41.085189 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:41.086432 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:41.087024 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:41.088554 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:41.088784 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:41.088868 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:41.091522 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:41.091525 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:41.091527 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:41.091552 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:41.092844 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:41.093315 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:41.093319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:41.093321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:41.093395 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:41.093755 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:41.093758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:41.093760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:41.093942 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:41.094024 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:41.094549 (+ 525us) server_negotiation.cc:299] Negotiation successful
0807 05:09:41.094690 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:09:41.095736 14295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:41.083063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:37493 (local address 127.8.16.66:45079)
0807 05:09:41.083518 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:41.083560 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:41.084864 (+ 1304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:41.085367 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:41.085374 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:41.085754 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:41.086297 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:41.086311 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:41.087150 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:41.087152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:41.088436 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:41.088448 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:41.088923 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:41.088926 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:41.089085 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:41.089688 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:41.089709 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:41.091357 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:41.092947 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:41.092953 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:41.092969 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:41.093223 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:41.093489 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:41.093492 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:41.093494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:41.093642 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:41.094040 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:41.094046 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:41.094302 (+ 256us) client_negotiation.cc:769] Sending connection context
0807 05:09:41.094481 (+ 179us) client_negotiation.cc:240] Negotiation successful
0807 05:09:41.094798 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":111,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.67@KRBTEST.COM" created.
I20260807 05:09:41.096937 14293 heartbeater.cc:347] Connected to a master server at 127.8.16.126:37493
I20260807 05:09:41.097229 14293 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:41.097823 14293 heartbeater.cc:511] Master 127.8.16.126:37493 requested a full tablet report, sending...
I20260807 05:09:41.099016 13953 ts_manager.cc:194] Registered new tserver with Master: 7a0540eb25fc44d792255eab52714611 (127.8.16.66:46297)
I20260807 05:09:41.099761 13953 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:45079
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
I20260807 05:09:41.110385 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37493
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:44095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:41.220099 14298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:41.220367 14298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:41.220410 14298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:41.224341 14298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:41.224395 14298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:41.224452 14298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:09:41.229544 14298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:44095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:41.230528 14298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:41.231518 14298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:41.240831 14304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:41.240834 14305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:41.240831 14308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:41.241071 14298 server_base.cc:1034] running on GCE node
I20260807 05:09:41.241489 14298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:41.242071 14298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:41.243227 14298 hybrid_clock.cc:648] HybridClock initialized: now 1786079381243204 us; error 41 us; skew 500 ppm
Aug 07 05:09:41 dist-test-slave-49h3 krb5kdc[13906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079381, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:41.245781 14298 init.cc:377] Logged in from keytab as kudu/127.8.16.67@KRBTEST.COM (short username kudu)
I20260807 05:09:41.246583 14298 webserver.cc:460] Webserver started at http://127.8.16.67:44279/ using document root <none> and password file <none>
I20260807 05:09:41.247071 14298 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:41.247125 14298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:41.247290 14298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:41.249212 14298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "482d5a4a836a4c5482a702ab8277ee55"
format_stamp: "Formatted at 2026-08-07 05:09:41 on dist-test-slave-49h3"
server_key: "4983267e9eac76e3178df84f99aacea5"
server_key_iv: "bd6408413494460067f7fe253e8e28c6"
server_key_version: "encryptionkey@0"
I20260807 05:09:41.249754 14298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "482d5a4a836a4c5482a702ab8277ee55"
format_stamp: "Formatted at 2026-08-07 05:09:41 on dist-test-slave-49h3"
server_key: "4983267e9eac76e3178df84f99aacea5"
server_key_iv: "bd6408413494460067f7fe253e8e28c6"
server_key_version: "encryptionkey@0"
I20260807 05:09:41.253212 14298 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:09:41.255520 14318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:41.256675 14298 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:41.256768 14298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "482d5a4a836a4c5482a702ab8277ee55"
format_stamp: "Formatted at 2026-08-07 05:09:41 on dist-test-slave-49h3"
server_key: "4983267e9eac76e3178df84f99aacea5"
server_key_iv: "bd6408413494460067f7fe253e8e28c6"
server_key_version: "encryptionkey@0"
I20260807 05:09:41.256850 14298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:41.283143 14298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:41.286108 14298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:41.286271 14298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:41.286855 14298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:41.287681 14298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:41.287730 14298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:41.287770 14298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:41.287786 14298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:41.296176 14298 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:39255
I20260807 05:09:41.296214 14438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:39255 every 8 connection(s)
I20260807 05:09:41.297046 14298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:09:41.306910 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 14298
I20260807 05:09:41.307024 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
Aug 07 05:09:41 dist-test-slave-49h3 krb5kdc[13906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079381, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:41.307334 8257 external_mini_cluster.cc:1428] Setting key 63a90c54b4865cc93da7d265b380e48f
I20260807 05:09:41.311270 13994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:41.298811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:54321 (local address 127.8.16.126:37493)
0807 05:09:41.298984 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:41.298988 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:41.299855 (+ 867us) server_negotiation.cc:407] Connection header received
0807 05:09:41.301407 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:41.301411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:41.301492 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:41.301589 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:41.303206 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:41.303862 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:41.304997 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:41.305234 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:41.305323 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:41.307812 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:41.307816 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:41.307819 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:41.307846 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:41.309563 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:41.310013 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:41.310016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:41.310017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:41.310085 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:41.310369 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:41.310370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:41.310371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:41.310535 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:41.310604 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:41.311043 (+ 439us) server_negotiation.cc:299] Negotiation successful
0807 05:09:41.311151 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 05:09:41.312069 14441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:41.299206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:37493 (local address 127.8.16.67:54321)
0807 05:09:41.299689 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:41.299735 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:41.301143 (+ 1408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:41.301802 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:41.301812 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:41.302324 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:41.303033 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:41.303051 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:41.303981 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:41.303984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:41.304865 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:41.304874 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:41.305379 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:41.305382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:41.305571 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:41.306118 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:41.306134 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:41.307647 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:41.309665 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:41.309670 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:41.309683 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:41.309939 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:41.310168 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:41.310170 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:41.310171 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:41.310304 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:41.310612 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:41.310627 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:41.310862 (+ 235us) client_negotiation.cc:769] Sending connection context
0807 05:09:41.311015 (+ 153us) client_negotiation.cc:240] Negotiation successful
0807 05:09:41.311266 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":116,"threads_started":1}
I20260807 05:09:41.313138 14439 heartbeater.cc:347] Connected to a master server at 127.8.16.126:37493
I20260807 05:09:41.313378 14439 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:41.313868 14439 heartbeater.cc:511] Master 127.8.16.126:37493 requested a full tablet report, sending...
I20260807 05:09:41.314857 13953 ts_manager.cc:194] Registered new tserver with Master: 482d5a4a836a4c5482a702ab8277ee55 (127.8.16.67:39255)
I20260807 05:09:41.315387 13953 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:54321
I20260807 05:09:41.321293 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/test-admin.keytab.
Aug 07 05:09:41 dist-test-slave-49h3 krb5kdc[13906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:41.338169 8257 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 05:09:41.341041 14446 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:41.339517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54380 (local address 127.8.16.65:44311)
0807 05:09:41.339792 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:41.339810 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:41.339840 (+ 30us) server_negotiation.cc:407] Connection header received
0807 05:09:41.339901 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:41.339907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:41.340096 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:41.340260 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:41.340873 (+ 613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54380: BlockingRecv error: recv got EOF from 127.0.0.1:54380 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":68,"threads_started":1}
W20260807 05:09:41.341133 8257 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/krb5cc' of type FILE
Aug 07 05:09:41 dist-test-slave-49h3 krb5kdc[13906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:41.342214 8257 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 05:09:41 dist-test-slave-49h3 krb5kdc[13906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:41.343029 8257 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 05:09:41 dist-test-slave-49h3 krb5kdc[13906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.65@KRBTEST.COM
I20260807 05:09:41.352510 14446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:41.343884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54394 (local address 127.8.16.65:44311)
0807 05:09:41.344002 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:41.344006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:41.344068 (+ 62us) server_negotiation.cc:407] Connection header received
0807 05:09:41.344208 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:41.344211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:41.344250 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:41.344311 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:41.344835 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:41.345724 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:41.346563 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:41.346949 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:41.347048 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:41.349266 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:41.349276 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:41.349290 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:41.349314 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:41.350776 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:41.351195 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:41.351200 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:41.351201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:41.351281 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:41.351526 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:41.351529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:41.351531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:41.351778 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:41.351973 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:41.352247 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 05:09:41.352378 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260807 05:09:41.356588 14446 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:41.355645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54408 (local address 127.8.16.65:44311)
0807 05:09:41.355760 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:41.355763 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:41.355916 (+ 153us) server_negotiation.cc:407] Connection header received
0807 05:09:41.356007 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:41.356009 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:41.356045 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:41.356113 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:41.356475 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54408: BlockingRecv error: recv got EOF from 127.0.0.1:54408 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 05:09:41.356870 8257 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/krb5cc' of type FILE
Aug 07 05:09:41 dist-test-slave-49h3 krb5kdc[13906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:41.357969 8257 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 05:09:41 dist-test-slave-49h3 krb5kdc[13906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:41.358829 8257 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 05:09:41 dist-test-slave-49h3 krb5kdc[13906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.65@KRBTEST.COM
I20260807 05:09:41.368232 14446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:41.359699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54414 (local address 127.8.16.65:44311)
0807 05:09:41.359847 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:41.359850 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:41.359908 (+ 58us) server_negotiation.cc:407] Connection header received
0807 05:09:41.360016 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:41.360018 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:41.360059 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:41.360156 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:41.360656 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:41.361335 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:41.362215 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:41.362535 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:41.362723 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:41.365213 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:41.365216 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:41.365218 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:41.365247 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:41.366677 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:41.367125 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:41.367128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:41.367129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:41.367239 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:41.367488 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:41.367490 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:41.367491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:41.367661 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:41.367753 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:41.367928 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 05:09:41.368035 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260807 05:09:41.371130 14446 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:41.370114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54426 (local address 127.8.16.65:44311)
0807 05:09:41.370286 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:41.370291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:41.370359 (+ 68us) server_negotiation.cc:407] Connection header received
0807 05:09:41.370496 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:41.370498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:41.370538 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:41.370602 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:41.371044 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54426: BlockingRecv error: recv got EOF from 127.0.0.1:54426 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260807 05:09:41.371299 8257 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079354071073-8257-0/krb5kdc/krb5cc' of type FILE
Aug 07 05:09:41 dist-test-slave-49h3 krb5kdc[13906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:41.372352 8257 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 05:09:41 dist-test-slave-49h3 krb5kdc[13906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:41.373239 8257 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 05:09:41 dist-test-slave-49h3 krb5kdc[13906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.65@KRBTEST.COM
I20260807 05:09:41.381002 14446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:41.374096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54440 (local address 127.8.16.65:44311)
0807 05:09:41.374203 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:41.374207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:41.374309 (+ 102us) server_negotiation.cc:407] Connection header received
0807 05:09:41.374432 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:41.374435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:41.374472 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:41.374528 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:41.375010 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:41.375507 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:41.376256 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:41.376470 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:41.376554 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:41.378562 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:41.378565 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:41.378567 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:41.378588 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:41.379624 (+ 1036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:41.379988 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:41.379991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:41.379993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:41.380063 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:41.380296 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:41.380299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:41.380300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:41.380447 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:41.380553 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:41.380766 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 05:09:41.380860 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 05:09:41.381911 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 14018
I20260807 05:09:41.391162 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 14158
I20260807 05:09:41.396834 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 14298
I20260807 05:09:41.403457 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 13919
2026-08-07T05:09:41Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2069 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:09:41 dist-test-slave-49h3 krb5kdc[14460](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:09:41 dist-test-slave-49h3 krb5kdc[14460](info): set up 2 sockets
Aug 07 05:09:41 dist-test-slave-49h3 krb5kdc[14460](info): commencing operation
krb5kdc: starting...
W20260807 05:09:42.381664 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:09:42 dist-test-slave-49h3 krb5kdc[14460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:09:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.126.keytab.
Entry for principal oryx/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.126.keytab.
I20260807 05:09:42.498302 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:46471
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.126.keytab
--principal=oryx/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:46471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:42.605861 14473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:42.606122 14473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:42.606161 14473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:42.610008 14473 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:42.610055 14473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:42.610072 14473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:42.610087 14473 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:42.610101 14473 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:42.615387 14473 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:46471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.126.keytab
--principal=oryx/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:46471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:42.616299 14473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:42.617231 14473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:42.625141 14481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:42.625154 14479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:42.625221 14473 server_base.cc:1034] running on GCE node
W20260807 05:09:42.625149 14483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:42.625949 14473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:42.627014 14473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:42.628170 14473 hybrid_clock.cc:648] HybridClock initialized: now 1786079382628151 us; error 32 us; skew 500 ppm
Aug 07 05:09:42 dist-test-slave-49h3 krb5kdc[14460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079382, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:42.630822 14473 init.cc:377] Logged in from keytab as oryx/127.8.16.126@KRBTEST.COM (short username oryx)
I20260807 05:09:42.631673 14473 webserver.cc:460] Webserver started at http://127.8.16.126:45713/ using document root <none> and password file <none>
I20260807 05:09:42.632179 14473 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:42.632246 14473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:42.632432 14473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:42.634470 14473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "81f0ce27c75145b3ace095da2ddb85d4"
format_stamp: "Formatted at 2026-08-07 05:09:42 on dist-test-slave-49h3"
server_key: "05d8ee8e20a4fd427ca6a2976d7b5854"
server_key_iv: "076e27316476879adc0e1f8f6b8d7765"
server_key_version: "encryptionkey@0"
I20260807 05:09:42.635003 14473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "81f0ce27c75145b3ace095da2ddb85d4"
format_stamp: "Formatted at 2026-08-07 05:09:42 on dist-test-slave-49h3"
server_key: "05d8ee8e20a4fd427ca6a2976d7b5854"
server_key_iv: "076e27316476879adc0e1f8f6b8d7765"
server_key_version: "encryptionkey@0"
I20260807 05:09:42.638514 14473 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:09:42.640720 14490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:42.641969 14473 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:09:42.642112 14473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "81f0ce27c75145b3ace095da2ddb85d4"
format_stamp: "Formatted at 2026-08-07 05:09:42 on dist-test-slave-49h3"
server_key: "05d8ee8e20a4fd427ca6a2976d7b5854"
server_key_iv: "076e27316476879adc0e1f8f6b8d7765"
server_key_version: "encryptionkey@0"
I20260807 05:09:42.642205 14473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:42.659503 14473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:42.662667 14473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:42.662850 14473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:42.669543 14473 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:46471
I20260807 05:09:42.669818 14543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:46471 every 8 connection(s)
I20260807 05:09:42.670619 14473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:42.673265 14544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:42.674299 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 14473
I20260807 05:09:42.674434 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:09:42.674733 8257 external_mini_cluster.cc:1428] Setting key 2ff2c4a40a8ed768568c88bd4751727e
I20260807 05:09:42.679514 14544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4: Bootstrap starting.
I20260807 05:09:42.681983 14544 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:42.682682 14544 log.cc:824] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4: Log is configured to *not* fsync() on all Append() calls
Aug 07 05:09:42 dist-test-slave-49h3 krb5kdc[14460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM
I20260807 05:09:42.684634 14544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4: No bootstrap required, opened a new log
I20260807 05:09:42.688897 14544 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } }
I20260807 05:09:42.689147 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:42.676085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39588 (local address 127.8.16.126:46471)
0807 05:09:42.676504 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:42.676523 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:42.676565 (+ 42us) server_negotiation.cc:407] Connection header received
0807 05:09:42.677347 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:42.677369 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:42.677791 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:42.678152 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:42.678797 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:42.680192 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:42.681047 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:42.681730 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:42.681895 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:42.684277 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:42.684283 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:42.684298 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:42.684329 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:42.686250 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:42.686672 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:42.686678 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:42.686685 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:42.686832 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:42.687087 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:42.687090 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:42.687092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:42.687563 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:42.687759 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:42.687940 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 05:09:42.688231 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":104,"threads_started":1}
I20260807 05:09:42.689162 14544 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:42.689242 14544 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81f0ce27c75145b3ace095da2ddb85d4, State: Initialized, Role: FOLLOWER
I20260807 05:09:42.689667 14544 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } }
I20260807 05:09:42.689802 14544 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:09:42.689868 14544 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:09:42.689962 14544 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:42.690779 14544 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } }
I20260807 05:09:42.691089 14544 leader_election.cc:302] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81f0ce27c75145b3ace095da2ddb85d4; no voters:
I20260807 05:09:42.691383 14544 leader_election.cc:288] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:09:42.691452 14551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:42.691643 14551 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [term 1 LEADER]: Becoming Leader. State: Replica: 81f0ce27c75145b3ace095da2ddb85d4, State: Running, Role: LEADER
I20260807 05:09:42.691946 14551 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } }
I20260807 05:09:42.692461 14544 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:42.693012 14553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81f0ce27c75145b3ace095da2ddb85d4. Latest consensus state: current_term: 1 leader_uuid: "81f0ce27c75145b3ace095da2ddb85d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } }
I20260807 05:09:42.693131 14553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:42.693534 14557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:42.693795 14552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81f0ce27c75145b3ace095da2ddb85d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } }
I20260807 05:09:42.693964 14552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:42.696234 14557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:42.701292 14557 catalog_manager.cc:1347] Generated new cluster ID: 5f5f89a5f3fa42ecb10b2f77a3d886cc
I20260807 05:09:42.701357 14557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:42.711118 14557 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:09:42.711969 14557 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:42.716120 14557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4: Generated new TSK 0
I20260807 05:09:42.716747 14557 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.65.keytab.
Entry for principal oryx/127.8.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.65.keytab.
I20260807 05:09:42.748549 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:46471
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.65.keytab
--principal=oryx/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:42.858273 14575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:42.858536 14575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:42.858596 14575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:42.862517 14575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:42.862592 14575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:42.862672 14575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:09:42.867877 14575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.65.keytab
--principal=oryx/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:46471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:42.868891 14575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:42.869925 14575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:42.878966 14583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:42.878988 14581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:42.878990 14580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:42.879257 14575 server_base.cc:1034] running on GCE node
I20260807 05:09:42.879628 14575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:42.880206 14575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:42.881376 14575 hybrid_clock.cc:648] HybridClock initialized: now 1786079382881361 us; error 27 us; skew 500 ppm
Aug 07 05:09:42 dist-test-slave-49h3 krb5kdc[14460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079382, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:42.884073 14575 init.cc:377] Logged in from keytab as oryx/127.8.16.65@KRBTEST.COM (short username oryx)
I20260807 05:09:42.884881 14575 webserver.cc:460] Webserver started at http://127.8.16.65:42179/ using document root <none> and password file <none>
I20260807 05:09:42.885366 14575 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:42.885421 14575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:42.885632 14575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:42.887526 14575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "fcf7f715e0ba4d5690059e682961cf0c"
format_stamp: "Formatted at 2026-08-07 05:09:42 on dist-test-slave-49h3"
server_key: "dbeea2f34fc329bd9f3f900b66b5fc25"
server_key_iv: "fdc042aad1200e8a485b5259a364b18c"
server_key_version: "encryptionkey@0"
I20260807 05:09:42.888024 14575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "fcf7f715e0ba4d5690059e682961cf0c"
format_stamp: "Formatted at 2026-08-07 05:09:42 on dist-test-slave-49h3"
server_key: "dbeea2f34fc329bd9f3f900b66b5fc25"
server_key_iv: "fdc042aad1200e8a485b5259a364b18c"
server_key_version: "encryptionkey@0"
I20260807 05:09:42.891508 14575 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:09:42.893865 14590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:42.895146 14575 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:42.895246 14575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "fcf7f715e0ba4d5690059e682961cf0c"
format_stamp: "Formatted at 2026-08-07 05:09:42 on dist-test-slave-49h3"
server_key: "dbeea2f34fc329bd9f3f900b66b5fc25"
server_key_iv: "fdc042aad1200e8a485b5259a364b18c"
server_key_version: "encryptionkey@0"
I20260807 05:09:42.895344 14575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:42.907176 14575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:42.910099 14575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:42.910267 14575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:42.910808 14575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:42.911554 14575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:42.911609 14575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:42.911649 14575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:42.911676 14575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:42.919675 14575 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:35079
I20260807 05:09:42.919703 14708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:35079 every 8 connection(s)
I20260807 05:09:42.920463 14575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:09:42.925402 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 14575
I20260807 05:09:42.925551 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:09:42.925833 8257 external_mini_cluster.cc:1428] Setting key f1c488d965e90397b515ba214c9fd60f
Aug 07 05:09:42 dist-test-slave-49h3 krb5kdc[14460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079382, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM
I20260807 05:09:42.934023 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:42.922130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:58709 (local address 127.8.16.126:46471)
0807 05:09:42.922286 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:42.922290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:42.923183 (+ 893us) server_negotiation.cc:407] Connection header received
0807 05:09:42.924023 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:42.924026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:42.924086 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:42.924189 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:42.925387 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:42.926023 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:42.927302 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:42.927533 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:42.927621 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:42.930391 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:42.930395 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:42.930397 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:42.930427 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:42.931985 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:42.932485 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:42.932488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:42.932491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:42.932584 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:42.932878 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:42.932881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:42.932883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:42.933118 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:42.933219 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:42.933726 (+ 507us) server_negotiation.cc:299] Negotiation successful
0807 05:09:42.933853 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 05:09:42.934832 14711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:42.922417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.8.16.65:58709)
0807 05:09:42.923044 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:42.923081 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:42.923814 (+ 733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:42.924380 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:42.924387 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:42.924754 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:42.925240 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:42.925255 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:42.926158 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:42.926161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:42.927186 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:42.927197 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:42.927669 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:42.927673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:42.927853 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:42.928407 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:42.928444 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:42.930235 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:42.932118 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:42.932123 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:42.932135 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:42.932384 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:42.932668 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:42.932670 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:42.932671 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:42.932796 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:42.933208 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:42.933213 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:42.933474 (+ 261us) client_negotiation.cc:769] Sending connection context
0807 05:09:42.933657 (+ 183us) client_negotiation.cc:240] Negotiation successful
0807 05:09:42.933910 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for oryx/127.8.16.66@KRBTEST.COM; defaulting to no policy
I20260807 05:09:42.936123 14709 heartbeater.cc:347] Connected to a master server at 127.8.16.126:46471
I20260807 05:09:42.936344 14709 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:42.936820 14709 heartbeater.cc:511] Master 127.8.16.126:46471 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.16.66@KRBTEST.COM" created.
I20260807 05:09:42.938431 14507 ts_manager.cc:194] Registered new tserver with Master: fcf7f715e0ba4d5690059e682961cf0c (127.8.16.65:35079)
I20260807 05:09:42.939870 14507 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:58709
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.66.keytab.
Entry for principal oryx/127.8.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.66.keytab.
I20260807 05:09:42.952595 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:46471
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.66.keytab
--principal=oryx/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:43.060262 14714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:43.060544 14714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:43.060607 14714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:43.064617 14714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:43.064692 14714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:43.064774 14714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:09:43.070096 14714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.66.keytab
--principal=oryx/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:46471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:43.071169 14714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:43.072181 14714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:43.081499 14719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:43.081614 14714 server_base.cc:1034] running on GCE node
W20260807 05:09:43.081499 14722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:43.081537 14720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:43.082216 14714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:43.082840 14714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:43.084019 14714 hybrid_clock.cc:648] HybridClock initialized: now 1786079383084002 us; error 28 us; skew 500 ppm
Aug 07 05:09:43 dist-test-slave-49h3 krb5kdc[14460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079383, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:43.086752 14714 init.cc:377] Logged in from keytab as oryx/127.8.16.66@KRBTEST.COM (short username oryx)
I20260807 05:09:43.087558 14714 webserver.cc:460] Webserver started at http://127.8.16.66:38999/ using document root <none> and password file <none>
I20260807 05:09:43.088078 14714 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:43.088133 14714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:43.088294 14714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:43.090265 14714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "795c4878987947afa5d8ab2c58773e59"
format_stamp: "Formatted at 2026-08-07 05:09:43 on dist-test-slave-49h3"
server_key: "85c32836cc01e553dc395d99372a6185"
server_key_iv: "9cdbe591f330709dc68e56370a717435"
server_key_version: "encryptionkey@0"
I20260807 05:09:43.090772 14714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "795c4878987947afa5d8ab2c58773e59"
format_stamp: "Formatted at 2026-08-07 05:09:43 on dist-test-slave-49h3"
server_key: "85c32836cc01e553dc395d99372a6185"
server_key_iv: "9cdbe591f330709dc68e56370a717435"
server_key_version: "encryptionkey@0"
I20260807 05:09:43.094370 14714 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 05:09:43.096544 14729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:43.097803 14714 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260807 05:09:43.097932 14714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "795c4878987947afa5d8ab2c58773e59"
format_stamp: "Formatted at 2026-08-07 05:09:43 on dist-test-slave-49h3"
server_key: "85c32836cc01e553dc395d99372a6185"
server_key_iv: "9cdbe591f330709dc68e56370a717435"
server_key_version: "encryptionkey@0"
I20260807 05:09:43.098023 14714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:43.124697 14714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:43.127735 14714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:43.127924 14714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:43.128535 14714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:43.129354 14714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:43.129403 14714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:43.129488 14714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:43.129515 14714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:43.138051 14714 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:37209
I20260807 05:09:43.138094 14843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:37209 every 8 connection(s)
I20260807 05:09:43.138928 14714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
Aug 07 05:09:43 dist-test-slave-49h3 krb5kdc[14460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079383, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM
I20260807 05:09:43.149138 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 14714
I20260807 05:09:43.149250 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:09:43.149631 8257 external_mini_cluster.cc:1428] Setting key afe9021ce62bcf79f61377b31d004baf
I20260807 05:09:43.152197 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.140721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:55363 (local address 127.8.16.126:46471)
0807 05:09:43.140842 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:43.140845 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:43.141914 (+ 1069us) server_negotiation.cc:407] Connection header received
0807 05:09:43.142783 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:43.142787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:43.142845 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:43.142918 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:43.144051 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.144568 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.145797 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.146055 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.146158 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.148568 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:43.148571 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:43.148574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:43.148610 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:43.150040 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.150593 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.150596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.150597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.150673 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.150993 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.150996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.150998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.151198 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:43.151276 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:43.151965 (+ 689us) server_negotiation.cc:299] Negotiation successful
0807 05:09:43.152079 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 05:09:43.153247 14846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.141070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.8.16.66:55363)
0807 05:09:43.141762 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:43.141803 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:43.142589 (+ 786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:43.143095 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:43.143101 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:43.143451 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:43.143914 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.143927 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.144693 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.144696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.145689 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.145699 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.146223 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.146228 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.146384 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.146933 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:43.146949 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:43.148416 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:43.150157 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.150162 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.150178 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.150487 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.150753 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.150756 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.150758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.150910 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.151325 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:43.151332 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:43.151659 (+ 327us) client_negotiation.cc:769] Sending connection context
0807 05:09:43.151892 (+ 233us) client_negotiation.cc:240] Negotiation successful
0807 05:09:43.152235 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":116,"threads_started":1}
I20260807 05:09:43.154408 14844 heartbeater.cc:347] Connected to a master server at 127.8.16.126:46471
I20260807 05:09:43.154652 14844 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:43.155195 14844 heartbeater.cc:511] Master 127.8.16.126:46471 requested a full tablet report, sending...
I20260807 05:09:43.156262 14507 ts_manager.cc:194] Registered new tserver with Master: 795c4878987947afa5d8ab2c58773e59 (127.8.16.66:37209)
I20260807 05:09:43.156826 14507 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:55363
WARNING: no policy specified for oryx/127.8.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.67.keytab.
Entry for principal oryx/127.8.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.67.keytab.
I20260807 05:09:43.176308 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:46471
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.67.keytab
--principal=oryx/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:43.283392 14850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:43.283656 14850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:43.283711 14850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:43.287662 14850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:43.287721 14850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:43.287799 14850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:09:43.292946 14850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.67.keytab
--principal=oryx/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:46471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:43.294039 14850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:43.295081 14850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:43.304280 14856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:43.304280 14859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:43.304286 14857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:43.304466 14850 server_base.cc:1034] running on GCE node
I20260807 05:09:43.304943 14850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:43.305575 14850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:43.306736 14850 hybrid_clock.cc:648] HybridClock initialized: now 1786079383306715 us; error 31 us; skew 500 ppm
Aug 07 05:09:43 dist-test-slave-49h3 krb5kdc[14460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079383, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:43.309935 14850 init.cc:377] Logged in from keytab as oryx/127.8.16.67@KRBTEST.COM (short username oryx)
I20260807 05:09:43.310812 14850 webserver.cc:460] Webserver started at http://127.8.16.67:45311/ using document root <none> and password file <none>
I20260807 05:09:43.311337 14850 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:43.311404 14850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:43.311599 14850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:43.313553 14850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "aebc96dd8d22409caba2a0968488a3b8"
format_stamp: "Formatted at 2026-08-07 05:09:43 on dist-test-slave-49h3"
server_key: "1e684f7f3243ef94441a24190a305759"
server_key_iv: "dd732459123d3ea93e8c2468c094a481"
server_key_version: "encryptionkey@0"
I20260807 05:09:43.314077 14850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "aebc96dd8d22409caba2a0968488a3b8"
format_stamp: "Formatted at 2026-08-07 05:09:43 on dist-test-slave-49h3"
server_key: "1e684f7f3243ef94441a24190a305759"
server_key_iv: "dd732459123d3ea93e8c2468c094a481"
server_key_version: "encryptionkey@0"
I20260807 05:09:43.317435 14850 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:09:43.319682 14866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:43.320773 14850 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 05:09:43.320876 14850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "aebc96dd8d22409caba2a0968488a3b8"
format_stamp: "Formatted at 2026-08-07 05:09:43 on dist-test-slave-49h3"
server_key: "1e684f7f3243ef94441a24190a305759"
server_key_iv: "dd732459123d3ea93e8c2468c094a481"
server_key_version: "encryptionkey@0"
I20260807 05:09:43.320959 14850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:43.334061 14850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:43.336825 14850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:43.336994 14850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:43.337605 14850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:43.338376 14850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:43.338423 14850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:43.338474 14850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:43.338500 14850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:43.346778 14850 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:38737
I20260807 05:09:43.346808 14984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:38737 every 8 connection(s)
I20260807 05:09:43.347608 14850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:09:43.352015 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 14850
I20260807 05:09:43.352113 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:09:43.352355 8257 external_mini_cluster.cc:1428] Setting key 344265551869c5be6e300e33201a7d73
Aug 07 05:09:43 dist-test-slave-49h3 krb5kdc[14460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079383, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM
I20260807 05:09:43.360284 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.349246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:55369 (local address 127.8.16.126:46471)
0807 05:09:43.349369 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:43.349373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:43.350351 (+ 978us) server_negotiation.cc:407] Connection header received
0807 05:09:43.351379 (+ 1028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:43.351382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:43.351441 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:43.351518 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:43.352720 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.353250 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.354399 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.354628 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.354714 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.357169 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:43.357173 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:43.357175 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:43.357199 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:43.358388 (+ 1189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.358858 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.358862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.358863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.358937 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.359239 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.359242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.359244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.359416 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:43.359503 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:43.359991 (+ 488us) server_negotiation.cc:299] Negotiation successful
0807 05:09:43.360122 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 05:09:43.361044 14987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.349583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.8.16.67:55369)
0807 05:09:43.350195 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:43.350245 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:43.351150 (+ 905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:43.351693 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:43.351700 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:43.352041 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:43.352576 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.352591 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.353371 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.353374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.354275 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.354285 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.354763 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.354766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.354922 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.355475 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:43.355491 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:43.356978 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:43.358496 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.358502 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.358518 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.358768 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.359016 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.359019 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.359021 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.359156 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.359514 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:43.359521 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:43.359766 (+ 245us) client_negotiation.cc:769] Sending connection context
0807 05:09:43.359930 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 05:09:43.360239 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":111,"threads_started":1}
I20260807 05:09:43.362149 14985 heartbeater.cc:347] Connected to a master server at 127.8.16.126:46471
I20260807 05:09:43.362376 14985 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:43.362845 14985 heartbeater.cc:511] Master 127.8.16.126:46471 requested a full tablet report, sending...
I20260807 05:09:43.363827 14507 ts_manager.cc:194] Registered new tserver with Master: aebc96dd8d22409caba2a0968488a3b8 (127.8.16.67:38737)
I20260807 05:09:43.364411 14507 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:55369
I20260807 05:09:43.365913 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.16.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.16.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.16.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.125.keytab.
Entry for principal oryx/127.8.16.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.125.keytab.
I20260807 05:09:43.392059 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.125:38779
--webserver_interface=127.8.16.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.125.keytab
--principal=oryx/127.8.16.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:46471,127.8.16.125:38779
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:43.500293 14990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:43.500571 14990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:43.500626 14990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:43.504622 14990 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:43.504685 14990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:43.504712 14990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:43.504735 14990 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:43.504763 14990 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:43.509873 14990 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:46471,127.8.16.125:38779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.125.keytab
--principal=oryx/127.8.16.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.125:38779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.16.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
W20260807 05:09:43.510061 14990 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.8.16.126:46471,127.8.16.125:38779'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 05:09:43.510854 14990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:43.511802 14990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:43.519587 14999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:43.519630 14996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:43.519675 14990 server_base.cc:1034] running on GCE node
W20260807 05:09:43.519632 14997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:43.520346 14990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:43.521363 14990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:43.522522 14990 hybrid_clock.cc:648] HybridClock initialized: now 1786079383522505 us; error 29 us; skew 500 ppm
Aug 07 05:09:43 dist-test-slave-49h3 krb5kdc[14460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079383, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:43.525200 14990 init.cc:377] Logged in from keytab as oryx/127.8.16.125@KRBTEST.COM (short username oryx)
I20260807 05:09:43.526085 14990 webserver.cc:460] Webserver started at http://127.8.16.125:45353/ using document root <none> and password file <none>
I20260807 05:09:43.526587 14990 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:43.526652 14990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:43.526859 14990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:43.528800 14990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/data/instance:
uuid: "421e96d56e9f4523a75349dccff2cb5d"
format_stamp: "Formatted at 2026-08-07 05:09:43 on dist-test-slave-49h3"
server_key: "16a5d679ebe63d674691b7590f8037d9"
server_key_iv: "e81e9044af5e88135ade9c75e06023a3"
server_key_version: "encryptionkey@0"
I20260807 05:09:43.529321 14990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/wal/instance:
uuid: "421e96d56e9f4523a75349dccff2cb5d"
format_stamp: "Formatted at 2026-08-07 05:09:43 on dist-test-slave-49h3"
server_key: "16a5d679ebe63d674691b7590f8037d9"
server_key_iv: "e81e9044af5e88135ade9c75e06023a3"
server_key_version: "encryptionkey@0"
I20260807 05:09:43.532717 14990 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 05:09:43.534930 15006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:43.536024 14990 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:43.536139 14990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/wal
uuid: "421e96d56e9f4523a75349dccff2cb5d"
format_stamp: "Formatted at 2026-08-07 05:09:43 on dist-test-slave-49h3"
server_key: "16a5d679ebe63d674691b7590f8037d9"
server_key_iv: "e81e9044af5e88135ade9c75e06023a3"
server_key_version: "encryptionkey@0"
I20260807 05:09:43.536222 14990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:43.548821 14990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:43.551417 14990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:43.551584 14990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:43.557912 14990 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.125:38779
I20260807 05:09:43.558161 15061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.125:38779 every 8 connection(s)
I20260807 05:09:43.558907 14990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/data/info.pb
I20260807 05:09:43.561604 15062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:43.565092 15062 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:09:43.568670 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 14990
Aug 07 05:09:43 dist-test-slave-49h3 krb5kdc[14460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079383, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.125@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM
Aug 07 05:09:43 dist-test-slave-49h3 krb5kdc[14460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.16.125@KRBTEST.COM
I20260807 05:09:43.578398 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.566860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39600 (local address 127.8.16.126:46471)
0807 05:09:43.567000 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:43.567005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:43.567783 (+ 778us) server_negotiation.cc:407] Connection header received
0807 05:09:43.568635 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:43.568638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:43.568683 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:43.568755 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:43.570205 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.570792 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.571930 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.572138 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.572211 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.574927 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:43.574930 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:43.574932 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:43.574956 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:43.576268 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.576820 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.576823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.576824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.576898 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.577286 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.577289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.577290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.577547 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:43.577639 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:43.578118 (+ 479us) server_negotiation.cc:299] Negotiation successful
0807 05:09:43.578237 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 05:09:43.579252 15066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.569538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46526 (local address 127.8.16.125:38779)
0807 05:09:43.569853 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:43.569862 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:43.569882 (+ 20us) server_negotiation.cc:407] Connection header received
0807 05:09:43.569984 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:43.569988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:43.570121 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:43.570293 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:43.570922 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.571798 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.572795 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.573217 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.573303 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.575718 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:43.575724 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:43.575732 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:43.575752 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:43.577427 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.577825 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.577831 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.577833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.577918 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.578170 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.578173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.578175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.578391 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:43.578562 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:43.578880 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 05:09:43.579010 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":236,"spinlock_wait_cycles":896,"thread_start_us":134,"threads_started":1}
I20260807 05:09:43.579258 15064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.567211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.0.0.1:39600)
0807 05:09:43.567642 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:43.567683 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:43.568412 (+ 729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:43.568981 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:43.568990 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:43.569415 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:43.570063 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.570080 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.570936 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.570939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.571809 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.571818 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.572271 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.572275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.572472 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.573103 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:43.573119 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:43.574754 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:43.576386 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.576396 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.576406 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.576727 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.576983 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.576986 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.576988 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.577156 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.577649 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:43.577655 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:43.577892 (+ 237us) client_negotiation.cc:769] Sending connection context
0807 05:09:43.578059 (+ 167us) client_negotiation.cc:240] Negotiation successful
0807 05:09:43.578365 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":41,"thread_start_us":111,"threads_started":1}
I20260807 05:09:43.580566 15062 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 05:09:43 dist-test-slave-49h3 krb5kdc[14460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079383, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.125@KRBTEST.COM for oryx/127.8.16.125@KRBTEST.COM
I20260807 05:09:43.587029 15064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.580990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:38779 (local address 127.0.0.1:46530)
0807 05:09:43.581073 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:43.581085 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:43.581151 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:43.581334 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:43.581336 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:43.581478 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:43.581574 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.581579 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.582218 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.582220 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.582721 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.582727 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.583080 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.583082 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.583148 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.583486 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:43.583496 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:43.584532 (+ 1036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:43.585993 (+ 1461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.585995 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.585997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.586179 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.586368 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.586370 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.586371 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.586431 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.586683 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:43.586685 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:43.586750 (+ 65us) client_negotiation.cc:769] Sending connection context
0807 05:09:43.586817 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 05:09:43.586906 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 05:09:43.587069 15066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.581104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46530 (local address 127.8.16.125:38779)
0807 05:09:43.581189 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:43.581192 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:43.581203 (+ 11us) server_negotiation.cc:407] Connection header received
0807 05:09:43.581227 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:43.581229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:43.581260 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:43.581316 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:43.581644 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.582134 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.582797 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.583013 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.583096 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.584650 (+ 1554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:43.584653 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:43.584655 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:43.584674 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:43.585899 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.586244 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.586246 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.586248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.586310 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.586487 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.586489 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.586490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.586625 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:43.586689 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:43.586848 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 05:09:43.586956 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":26}
I20260807 05:09:43.589782 15062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d: Bootstrap starting.
I20260807 05:09:43.591877 15062 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:43.592577 15062 log.cc:824] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:43.594398 15062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d: No bootstrap required, opened a new log
I20260807 05:09:43.597218 15062 raft_consensus.cc:348] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } }
I20260807 05:09:43.597409 15062 raft_consensus.cc:374] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:43.597483 15062 raft_consensus.cc:729] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 421e96d56e9f4523a75349dccff2cb5d, State: Initialized, Role: FOLLOWER
I20260807 05:09:43.597877 15062 consensus_queue.cc:260] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } }
I20260807 05:09:43.598301 15068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } } }
I20260807 05:09:43.598414 15068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:09:43.598999 15062 sys_catalog.cc:564] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: configured and running, proceeding with master startup.
W20260807 05:09:43.603005 15079 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:09:43.603073 15079 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:09:43.605581 14990 master_runner.cc:428] Detected that this master 421e96d56e9f4523a75349dccff2cb5d is joining an existing cluster
I20260807 05:09:43.605643 14990 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.16.125:38779
I20260807 05:09:43.615855 15066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.607870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46546 (local address 127.8.16.125:38779)
0807 05:09:43.607965 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:43.607969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:43.608363 (+ 394us) server_negotiation.cc:407] Connection header received
0807 05:09:43.608458 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:43.608461 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:43.608496 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:43.608556 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:43.609114 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.609936 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.610752 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.611025 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.611107 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.612686 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:43.612690 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:43.612693 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:43.612716 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:43.614324 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.614774 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.614777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.614779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.614852 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.615119 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.615122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.615124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.615286 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:43.615368 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:43.615584 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 05:09:43.615727 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":85,"server-negotiator.queue_time_us":38}
I20260807 05:09:43.615868 15087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.607773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:38779 (local address 127.0.0.1:46546)
0807 05:09:43.608288 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:43.608301 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:43.608394 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:43.608646 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:43.608648 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:43.608895 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:43.609004 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.609011 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.610028 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.610031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.610647 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.610655 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.611129 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.611133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.611219 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.612113 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:43.612128 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:43.612565 (+ 437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:43.614436 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.614438 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.614441 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.614692 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.614934 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.614936 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.614938 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.615026 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.615371 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:43.615374 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:43.615453 (+ 79us) client_negotiation.cc:769] Sending connection context
0807 05:09:43.615535 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 05:09:43.615669 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":38,"thread_start_us":97,"threads_started":1}
I20260807 05:09:43.616161 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.607841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39606 (local address 127.8.16.126:46471)
0807 05:09:43.607954 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:43.607958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:43.608182 (+ 224us) server_negotiation.cc:407] Connection header received
0807 05:09:43.608361 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:43.608364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:43.608410 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:43.608478 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:43.608946 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.609823 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.611158 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.611491 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.611564 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.612934 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:43.612938 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:43.612941 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:43.612967 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:43.614420 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.614921 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.614925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.614927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.615007 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.615289 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.615293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.615295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.615497 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:43.615562 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:43.615897 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 05:09:43.616009 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 05:09:43.616649 15086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.607773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.0.0.1:39606)
0807 05:09:43.608102 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:43.608117 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:43.608247 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:43.608528 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:43.608531 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:43.608733 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:43.608841 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.608846 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.609938 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.609941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.611020 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.611031 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.611711 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.611714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.611816 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.612242 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:43.612257 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:43.612796 (+ 539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:43.614519 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.614523 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.614525 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.614822 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.615090 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.615093 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.615095 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.615181 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.615631 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:43.615634 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:43.615715 (+ 81us) client_negotiation.cc:769] Sending connection context
0807 05:09:43.616344 (+ 629us) client_negotiation.cc:240] Negotiation successful
0807 05:09:43.616479 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":67,"thread_start_us":104,"threads_started":1}
Aug 07 05:09:43 dist-test-slave-49h3 krb5kdc[14460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079382, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.126@KRBTEST.COM for oryx/127.8.16.125@KRBTEST.COM
I20260807 05:09:43.629289 15066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.620909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46562 (local address 127.8.16.125:38779)
0807 05:09:43.621019 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:43.621023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:43.621346 (+ 323us) server_negotiation.cc:407] Connection header received
0807 05:09:43.621550 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:43.621553 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:43.621604 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:43.621675 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:43.622258 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.623011 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.623877 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.624208 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.624305 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.626524 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:43.626527 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:43.626529 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:43.626550 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:43.627741 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.628229 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.628231 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.628232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.628307 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.628544 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.628546 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.628547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.628691 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:43.628762 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:43.629083 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 05:09:43.629165 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 05:09:43.629498 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.620863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:38779 (local address 127.0.0.1:46562)
0807 05:09:43.621218 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:43.621272 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:43.621401 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:43.621716 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:43.621720 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:43.622058 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:43.622164 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.622170 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.623090 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.623092 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.623777 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.623786 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.624303 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.624306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.624385 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.624907 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:43.624918 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:43.626396 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:43.627841 (+ 1445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.627851 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.627857 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.628146 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.628377 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.628380 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.628382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.628473 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.628777 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:43.628784 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:43.628906 (+ 122us) client_negotiation.cc:769] Sending connection context
0807 05:09:43.629048 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 05:09:43.629230 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":89,"threads_started":1}
I20260807 05:09:43.630760 14507 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.16.125:38779: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d"
member_type: NON_VOTER
last_known_addr {
host: "127.8.16.125"
port: 38779
}
attrs {
promote: true
}
}
dest_uuid: "81f0ce27c75145b3ace095da2ddb85d4"
cas_config_opid_index: -1
I20260807 05:09:43.631493 14507 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: NON_VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: true } }
I20260807 05:09:43.633965 15034 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:43.635429 15034 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 1 FOLLOWER]: Refusing update from remote peer 81f0ce27c75145b3ace095da2ddb85d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 05:09:43.633421 15089 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 421e96d56e9f4523a75349dccff2cb5d (127.8.16.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: NON_VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: true } } }
I20260807 05:09:43.636039 15090 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: NON_VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:43.637002 15089 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.16.125:38779
I20260807 05:09:43.641880 14990 master.cc:537] Master@127.8.16.125:38779 shutting down...
I20260807 05:09:43.642148 15034 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 421e96d56e9f4523a75349dccff2cb5d (127.8.16.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: NON_VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: true } } }
I20260807 05:09:43.643028 15093 mvcc.cc:205] Tried to move back new op lower bound from 7315781151601696768 to 7315781151507066880. Current Snapshot: MvccSnapshot[applied={T|T < 7315781151536095232}]
I20260807 05:09:43.644423 15068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81f0ce27c75145b3ace095da2ddb85d4. Latest consensus state: current_term: 1 leader_uuid: "81f0ce27c75145b3ace095da2ddb85d4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: NON_VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: true } } }
I20260807 05:09:43.644531 15068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: This master's current role is: LEARNER
I20260807 05:09:43.644637 15068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "81f0ce27c75145b3ace095da2ddb85d4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: NON_VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: true } } }
I20260807 05:09:43.644716 15068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: This master's current role is: LEARNER
I20260807 05:09:43.644781 15068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "81f0ce27c75145b3ace095da2ddb85d4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: NON_VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: true } } }
I20260807 05:09:43.644840 15068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: This master's current role is: LEARNER
I20260807 05:09:43.645004 15090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "81f0ce27c75145b3ace095da2ddb85d4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: NON_VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: true } } }
I20260807 05:09:43.645124 15090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [sys.catalog]: This master's current role is: LEADER
WARNING: no policy specified for oryx/127.8.16.124@KRBTEST.COM; defaulting to no policy
I20260807 05:09:43.648419 14990 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 1 LEARNER]: Raft consensus shutting down.
I20260807 05:09:43.648694 14990 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 1 LEARNER]: Raft consensus is shut down!
I20260807 05:09:43.648779 14990 tablet_replica.cc:330] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.16.124@KRBTEST.COM" created.
I20260807 05:09:43.664317 14990 master.cc:552] Master@127.8.16.125:38779 shutdown complete.
I20260807 05:09:43.664376 14990 master_runner.cc:305] Clearing existing system tablet
I20260807 05:09:43.664619 14990 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:43.664676 14990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.16.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.124.keytab.
Entry for principal oryx/127.8.16.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.124.keytab.
I20260807 05:09:43.666046 14990 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 05:09:43.666023 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.124:44531
--webserver_interface=127.8.16.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.124.keytab
--principal=oryx/127.8.16.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:46471,127.8.16.125:38779,127.8.16.124:44531
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:09:43.666869 15103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:43.667066 14990 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:43.667198 14990 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 05:09:43.667253 14990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/wal
uuid: "421e96d56e9f4523a75349dccff2cb5d"
format_stamp: "Formatted at 2026-08-07 05:09:43 on dist-test-slave-49h3"
server_key: "16a5d679ebe63d674691b7590f8037d9"
server_key_iv: "e81e9044af5e88135ade9c75e06023a3"
server_key_version: "encryptionkey@0"
I20260807 05:09:43.669111 14990 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:43.670984 14990 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 05:09:43.671068 14990 log.cc:1196] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 05:09:43.671325 14990 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d: Deleting consensus metadata
I20260807 05:09:43.671475 14990 master_runner.cc:315] Copying system tablet from 127.8.16.126:46471
I20260807 05:09:43.672758 14990 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d: tablet copy: Beginning tablet copy session from remote peer at address 127.8.16.126:46471
I20260807 05:09:43.686054 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.673404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39618 (local address 127.8.16.126:46471)
0807 05:09:43.673633 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:43.673637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:43.677760 (+ 4123us) server_negotiation.cc:407] Connection header received
0807 05:09:43.677981 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:43.677985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:43.678041 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:43.678095 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:43.678784 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.679577 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.680782 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.681105 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.681183 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.682656 (+ 1473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:43.682660 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:43.682663 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:43.682688 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:43.684188 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.684662 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.684666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.684668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.684747 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.685114 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.685117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.685119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.685323 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:43.685388 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:43.685774 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 05:09:43.685891 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260807 05:09:43.686870 15111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.673744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.0.0.1:39618)
0807 05:09:43.677641 (+ 3897us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:43.677658 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:43.677857 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:43.678263 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:43.678266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:43.678482 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:43.678637 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.678646 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.679741 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.679745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.680639 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.680650 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.681355 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.681359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.681498 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.682068 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:43.682082 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:43.682494 (+ 412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:43.684304 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.684307 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.684309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.684548 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.684932 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.684934 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.684937 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.685022 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.685486 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:43.685489 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:43.685583 (+ 94us) client_negotiation.cc:769] Sending connection context
0807 05:09:43.686499 (+ 916us) client_negotiation.cc:240] Negotiation successful
0807 05:09:43.686659 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3800,"thread_start_us":88,"threads_started":1}
I20260807 05:09:43.687487 14527 tablet_copy_service.cc:140] P 81f0ce27c75145b3ace095da2ddb85d4: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 421e96d56e9f4523a75349dccff2cb5d ({username='oryx', principal='oryx/127.8.16.125@KRBTEST.COM'} at 127.0.0.1:39618)
I20260807 05:09:43.687669 14527 tablet_copy_service.cc:161] P 81f0ce27c75145b3ace095da2ddb85d4: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 421e96d56e9f4523a75349dccff2cb5d at {username='oryx', principal='oryx/127.8.16.125@KRBTEST.COM'} at 127.0.0.1:39618: session id = 421e96d56e9f4523a75349dccff2cb5d-00000000000000000000000000000000
I20260807 05:09:43.689620 14527 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 05:09:43.691426 14990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 05:09:43.693902 14990 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d: tablet copy: Starting download of 0 data blocks...
I20260807 05:09:43.694114 14990 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d: tablet copy: Starting download of 1 WAL segments...
I20260807 05:09:43.695945 14990 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 05:09:43.697777 14527 tablet_copy_service.cc:342] P 81f0ce27c75145b3ace095da2ddb85d4: Request end of tablet copy session 421e96d56e9f4523a75349dccff2cb5d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.16.125@KRBTEST.COM'} at 127.0.0.1:39618
I20260807 05:09:43.697913 14527 tablet_copy_service.cc:433] P 81f0ce27c75145b3ace095da2ddb85d4: ending tablet copy session 421e96d56e9f4523a75349dccff2cb5d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 421e96d56e9f4523a75349dccff2cb5d
I20260807 05:09:43.705615 14990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:43.707322 15112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:43.707330 15115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:43.707718 15113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:43.708235 14990 server_base.cc:1034] running on GCE node
I20260807 05:09:43.708385 14990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:43.708669 14990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:43.709800 14990 hybrid_clock.cc:648] HybridClock initialized: now 1786079383709788 us; error 27 us; skew 500 ppm
Aug 07 05:09:43 dist-test-slave-49h3 krb5kdc[14460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079383, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:43.711555 14990 init.cc:377] Logged in from keytab as oryx/127.8.16.125@KRBTEST.COM (short username oryx)
I20260807 05:09:43.712023 14990 webserver.cc:460] Webserver started at http://127.8.16.125:33343/ using document root <none> and password file <none>
I20260807 05:09:43.712200 14990 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:43.712231 14990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:43.713588 14990 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 05:09:43.714692 15122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:43.714951 14990 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 05:09:43.715060 14990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/wal
uuid: "421e96d56e9f4523a75349dccff2cb5d"
format_stamp: "Formatted at 2026-08-07 05:09:43 on dist-test-slave-49h3"
server_key: "16a5d679ebe63d674691b7590f8037d9"
server_key_iv: "e81e9044af5e88135ade9c75e06023a3"
server_key_version: "encryptionkey@0"
I20260807 05:09:43.715211 14990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:43.724390 14990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:43.726902 14990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:43.732592 14990 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.125:38779
I20260807 05:09:43.732623 15177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.125:38779 every 8 connection(s)
I20260807 05:09:43.733402 14990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-1/data/info.pb
I20260807 05:09:43.735297 15178 sys_catalog.cc:263] Verifying existing consensus state
I20260807 05:09:43.736295 15178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d: Bootstrap starting.
I20260807 05:09:43.743974 15178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:09:43.744263 15178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d: Bootstrap complete.
I20260807 05:09:43.744863 15178 raft_consensus.cc:348] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: NON_VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: true } }
I20260807 05:09:43.745043 15178 raft_consensus.cc:729] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 421e96d56e9f4523a75349dccff2cb5d, State: Initialized, Role: LEARNER
I20260807 05:09:43.745216 15178 consensus_queue.cc:260] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: NON_VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: true } }
I20260807 05:09:43.745597 15182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: NON_VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: true } } }
I20260807 05:09:43.745707 15178 sys_catalog.cc:564] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:43.745895 15182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: This master's current role is: LEARNER
I20260807 05:09:43.750075 15194 catalog_manager.cc:1259] Loaded cluster ID: 5f5f89a5f3fa42ecb10b2f77a3d886cc
I20260807 05:09:43.750136 15194 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d: loading cluster ID for follower catalog manager: success
I20260807 05:09:43.752142 15194 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d: acquiring CA information for follower catalog manager: success
I20260807 05:09:43.753470 15194 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 05:09:43.797873 15104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:43.798143 15104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:43.798202 15104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:43.802273 15104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:43.802331 15104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:43.802357 15104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:43.802380 15104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:43.802408 15104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:43.807545 15104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:46471,127.8.16.125:38779,127.8.16.124:44531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.124.keytab
--principal=oryx/127.8.16.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.124:44531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.16.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:43.808544 15104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:43.809552 15104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:43.817557 15202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:43.817550 15199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:43.817562 15200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:43.817787 15104 server_base.cc:1034] running on GCE node
I20260807 05:09:43.818385 15104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:43.819453 15104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:43.820603 15104 hybrid_clock.cc:648] HybridClock initialized: now 1786079383820586 us; error 29 us; skew 500 ppm
Aug 07 05:09:43 dist-test-slave-49h3 krb5kdc[14460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079383, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:43.823187 15104 init.cc:377] Logged in from keytab as oryx/127.8.16.124@KRBTEST.COM (short username oryx)
I20260807 05:09:43.824051 15104 webserver.cc:460] Webserver started at http://127.8.16.124:38373/ using document root <none> and password file <none>
I20260807 05:09:43.824571 15104 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:43.824636 15104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:43.824827 15104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:43.826967 15104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/data/instance:
uuid: "8b93668c0b5f46a8ad3b2c6c983d461d"
format_stamp: "Formatted at 2026-08-07 05:09:43 on dist-test-slave-49h3"
server_key: "ac29d53add3b42fc81b5304ea5c34bdd"
server_key_iv: "be2ff39f1d323d549ff2892925efa09c"
server_key_version: "encryptionkey@0"
I20260807 05:09:43.827515 15104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/wal/instance:
uuid: "8b93668c0b5f46a8ad3b2c6c983d461d"
format_stamp: "Formatted at 2026-08-07 05:09:43 on dist-test-slave-49h3"
server_key: "ac29d53add3b42fc81b5304ea5c34bdd"
server_key_iv: "be2ff39f1d323d549ff2892925efa09c"
server_key_version: "encryptionkey@0"
I20260807 05:09:43.831070 15104 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:09:43.833298 15209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:43.834414 15104 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:43.834523 15104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/wal
uuid: "8b93668c0b5f46a8ad3b2c6c983d461d"
format_stamp: "Formatted at 2026-08-07 05:09:43 on dist-test-slave-49h3"
server_key: "ac29d53add3b42fc81b5304ea5c34bdd"
server_key_iv: "be2ff39f1d323d549ff2892925efa09c"
server_key_version: "encryptionkey@0"
I20260807 05:09:43.834610 15104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:43.850031 15104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:43.852985 15104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:43.853165 15104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:43.860108 15104 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.124:44531
I20260807 05:09:43.860359 15263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.124:44531 every 8 connection(s)
I20260807 05:09:43.861037 15104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/data/info.pb
I20260807 05:09:43.863696 15264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:43.867394 15264 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:09:43.867707 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 15104
Aug 07 05:09:43 dist-test-slave-49h3 krb5kdc[14460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079383, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.124@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM
Aug 07 05:09:43 dist-test-slave-49h3 krb5kdc[14460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.16.124@KRBTEST.COM
I20260807 05:09:43.879539 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.869889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39634 (local address 127.8.16.126:46471)
0807 05:09:43.870018 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:43.870022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:43.870186 (+ 164us) server_negotiation.cc:407] Connection header received
0807 05:09:43.870574 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:43.870576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:43.870621 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:43.870688 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:43.871577 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.872096 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.873172 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.873392 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.873504 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.876015 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:43.876018 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:43.876020 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:43.876043 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:43.877423 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.878028 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.878031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.878033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.878109 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.878426 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.878429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.878431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.878629 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:43.878722 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:43.879252 (+ 530us) server_negotiation.cc:299] Negotiation successful
0807 05:09:43.879375 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 05:09:43.880499 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.869091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47830 (local address 127.8.16.124:44531)
0807 05:09:43.869502 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:43.869518 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:43.869551 (+ 33us) server_negotiation.cc:407] Connection header received
0807 05:09:43.870367 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:43.870388 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:43.870612 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:43.870772 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:43.871452 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.872412 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.873202 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.873657 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.873752 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.876392 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:43.876398 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:43.876411 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:43.876435 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:43.878087 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.878464 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.878469 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.878470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.878534 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.878803 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.878805 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.878807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.879075 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:43.879272 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:43.879471 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 05:09:43.879585 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":262,"thread_start_us":117,"threads_started":1}
I20260807 05:09:43.880499 15267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.869699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.0.0.1:39634)
0807 05:09:43.870032 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:43.870098 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:43.870391 (+ 293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:43.870761 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:43.870766 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:43.871187 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:43.871432 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.871450 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.872212 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.872215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.873042 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.873052 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.873572 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.873576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.873715 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.874368 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:43.874384 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:43.875884 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:43.877546 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.877559 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.877568 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.877940 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.878204 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.878207 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.878209 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.878341 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.878723 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:43.878728 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:43.879035 (+ 307us) client_negotiation.cc:769] Sending connection context
0807 05:09:43.879196 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 05:09:43.879503 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":32,"thread_start_us":66,"threads_started":1}
I20260807 05:09:43.881968 15264 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 05:09:43 dist-test-slave-49h3 krb5kdc[14460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079383, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.124@KRBTEST.COM for oryx/127.8.16.125@KRBTEST.COM
I20260807 05:09:43.889223 15267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.882411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:38779 (local address 127.0.0.1:46574)
0807 05:09:43.882547 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:43.882557 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:43.882612 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:43.883022 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:43.883024 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:43.883158 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:43.883251 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.883257 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.884131 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.884133 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.884645 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.884652 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.885073 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.885077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.885166 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.885586 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:43.885597 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:43.886717 (+ 1120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:43.888120 (+ 1403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.888122 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.888124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.888297 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.888513 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.888515 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.888516 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.888582 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.888880 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:43.888882 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:43.888949 (+ 67us) client_negotiation.cc:769] Sending connection context
0807 05:09:43.889017 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 05:09:43.889110 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 05:09:43.889276 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.882527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46574 (local address 127.8.16.125:38779)
0807 05:09:43.882830 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:43.882833 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:43.882845 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:09:43.882878 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:43.882880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:43.882918 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:43.882992 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:43.883368 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.884031 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.884756 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.884982 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.885067 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.886848 (+ 1781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:43.886851 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:43.886853 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:43.886876 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:43.888034 (+ 1158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.888366 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.888368 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.888369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.888434 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.888643 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.888645 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.888646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.888817 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:43.888888 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:43.889058 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 05:09:43.889150 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":129,"threads_started":1}
I20260807 05:09:43.889971 15264 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.124" port: 44531 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 05:09:43 dist-test-slave-49h3 krb5kdc[14460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079383, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.124@KRBTEST.COM for oryx/127.8.16.124@KRBTEST.COM
I20260807 05:09:43.897082 15267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.890407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:44531 (local address 127.0.0.1:47834)
0807 05:09:43.890492 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:43.890505 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:43.890575 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:43.890768 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:43.890771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:43.890900 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:43.890958 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.890963 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.891630 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.891633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.892142 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.892150 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.892952 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.892956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.893057 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.893379 (+ 322us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:43.893389 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:43.894497 (+ 1108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:43.895845 (+ 1348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.895848 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.895850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.896018 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.896235 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.896238 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.896240 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.896310 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.896616 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:43.896619 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:43.896696 (+ 77us) client_negotiation.cc:769] Sending connection context
0807 05:09:43.896782 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 05:09:43.896908 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260807 05:09:43.897089 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.890438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47834 (local address 127.8.16.124:44531)
0807 05:09:43.890529 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:43.890533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:43.890545 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:09:43.890644 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:43.890646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:43.890681 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:43.890741 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:43.891050 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.891537 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.892244 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.892839 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.892931 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.894609 (+ 1678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:43.894612 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:43.894615 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:43.894636 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:43.895756 (+ 1120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.896094 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.896097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.896099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.896162 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.896382 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.896384 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.896386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.896537 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:43.896615 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:43.896831 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 05:09:43.896961 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":37}
I20260807 05:09:43.900070 15264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d: Bootstrap starting.
I20260807 05:09:43.902491 15264 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:43.903182 15264 log.cc:824] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:43.905107 15264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d: No bootstrap required, opened a new log
I20260807 05:09:43.908160 15264 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 44531 } }
I20260807 05:09:43.908383 15264 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:43.908416 15264 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b93668c0b5f46a8ad3b2c6c983d461d, State: Initialized, Role: FOLLOWER
I20260807 05:09:43.908792 15264 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 44531 } }
I20260807 05:09:43.909272 15270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 44531 } } }
I20260807 05:09:43.909404 15270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:09:43.909950 15264 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [sys.catalog]: configured and running, proceeding with master startup.
W20260807 05:09:43.914470 15281 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:09:43.914553 15281 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:09:43.917876 15104 master_runner.cc:428] Detected that this master 8b93668c0b5f46a8ad3b2c6c983d461d is joining an existing cluster
I20260807 05:09:43.917935 15104 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.16.124:44531
I20260807 05:09:43.930411 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.920010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39650 (local address 127.8.16.126:46471)
0807 05:09:43.920140 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:43.920145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:43.922560 (+ 2415us) server_negotiation.cc:407] Connection header received
0807 05:09:43.922699 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:43.922702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:43.922755 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:43.922810 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:43.923427 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.924214 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.925550 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.925879 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.925947 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.927219 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:43.927222 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:43.927225 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:43.927250 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:43.928689 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.929122 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.929126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.929128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.929222 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.929541 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.929545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.929547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.929740 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:43.929829 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:43.930144 (+ 315us) server_negotiation.cc:299] Negotiation successful
0807 05:09:43.930272 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 05:09:43.930909 15290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.920591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.0.0.1:39650)
0807 05:09:43.922470 (+ 1879us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:43.922482 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:43.922576 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:43.922959 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:43.922961 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:43.923143 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:43.923309 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.923317 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.924420 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.924423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.925377 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.925388 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.926131 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.926134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.926217 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.926733 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:43.926745 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:43.927083 (+ 338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:43.928796 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.928799 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.928801 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.929001 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.929335 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.929338 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.929340 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.929437 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.929834 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:43.929837 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:43.929928 (+ 91us) client_negotiation.cc:769] Sending connection context
0807 05:09:43.930576 (+ 648us) client_negotiation.cc:240] Negotiation successful
0807 05:09:43.930727 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1818,"mutex_wait_us":21,"spinlock_wait_cycles":896,"thread_start_us":80,"threads_started":1}
I20260807 05:09:43.933887 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 14575
I20260807 05:09:43.937486 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.922353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46586 (local address 127.8.16.125:38779)
0807 05:09:43.922478 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:43.922482 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:43.925110 (+ 2628us) server_negotiation.cc:407] Connection header received
0807 05:09:43.925287 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:43.925290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:43.925363 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:43.925415 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:43.926149 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.926939 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.928284 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.928657 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.928767 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.929932 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:43.929936 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:43.929939 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:43.929964 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:43.931442 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.932485 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.932488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.932490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.932588 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.932909 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.932913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.932915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.933144 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:43.933216 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:43.933597 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 05:09:43.937279 (+ 3682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 05:09:43.938133 15289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.922224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:38779 (local address 127.0.0.1:46586)
0807 05:09:43.925024 (+ 2800us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:43.925037 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:43.925171 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:43.925628 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:43.925631 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:43.925874 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:43.925993 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.926000 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.927075 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.927078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.928126 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.928139 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.928771 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.928775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.928869 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.929327 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:43.929341 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:43.929813 (+ 472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:43.931545 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.931548 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.931550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.932393 (+ 843us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.932675 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.932678 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.932680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.932799 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.933289 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:43.933292 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:43.933379 (+ 87us) client_negotiation.cc:769] Sending connection context
0807 05:09:43.937722 (+ 4343us) client_negotiation.cc:240] Negotiation successful
0807 05:09:43.937945 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2737,"thread_start_us":86,"threads_started":1}
I20260807 05:09:43.944157 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:35079
--local_ip_for_outbound_sockets=127.8.16.65
--tserver_master_addrs=127.8.16.126:46471
--webserver_port=42179
--webserver_interface=127.8.16.65
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.65.keytab
--principal=oryx/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.16.126:46471,127.8.16.125:38779
--tserver_master_addrs=127.8.16.126:46471,127.8.16.125:38779,127.8.16.124:44531 with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 05:09:43 dist-test-slave-49h3 krb5kdc[14460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079382, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.126@KRBTEST.COM for oryx/127.8.16.124@KRBTEST.COM
I20260807 05:09:43.950856 15288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.920266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:44531 (local address 127.0.0.1:47842)
0807 05:09:43.920554 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:43.920569 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:43.920663 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:43.922677 (+ 2014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:43.922680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:43.922975 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:43.923120 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.923129 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.924123 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.924126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.925624 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.925633 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.926211 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.926213 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.926312 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.946144 (+ 19832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:43.946159 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:43.946604 (+ 445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:43.949349 (+ 2745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.949352 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.949354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.949622 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.949864 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.949867 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.949869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.949937 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.950354 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:43.950357 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:43.950431 (+ 74us) client_negotiation.cc:769] Sending connection context
0807 05:09:43.950513 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 05:09:43.950653 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":4,"spinlock_wait_cycles":2816,"thread_start_us":103,"threads_started":1}
I20260807 05:09:43.951206 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.936985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:44531 (local address 127.0.0.1:47858)
0807 05:09:43.941592 (+ 4607us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:43.941610 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:43.941715 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:43.942031 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:43.942035 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:43.942256 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:43.942376 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.942385 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.943489 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.943493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.944338 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.944350 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.945158 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.945162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.945267 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.945831 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:43.945846 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:43.947407 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:43.949543 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.949547 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.949549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.949814 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.950139 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.950143 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.950145 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.950241 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.950699 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:43.950702 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:43.950810 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 05:09:43.950894 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 05:09:43.951040 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604}
I20260807 05:09:43.951869 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.920265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47842 (local address 127.8.16.124:44531)
0807 05:09:43.922471 (+ 2206us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:43.922476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:43.922490 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:09:43.922524 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:43.922526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:43.922568 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:43.922647 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:43.923237 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.923983 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.925718 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.926004 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.926092 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.946769 (+ 20677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:43.946772 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:43.946775 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:43.946798 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:43.949242 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.949715 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.949718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.949720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.949788 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.950009 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.950012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.950014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.950244 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:43.950325 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:43.951563 (+ 1238us) server_negotiation.cc:299] Negotiation successful
0807 05:09:43.951693 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2146,"spinlock_wait_cycles":3840}
I20260807 05:09:43.951983 15291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.937142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47858 (local address 127.8.16.124:44531)
0807 05:09:43.937415 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:43.937423 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:43.941675 (+ 4252us) server_negotiation.cc:407] Connection header received
0807 05:09:43.941840 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:43.941843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:43.941898 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:43.941987 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:43.942517 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.943352 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.944612 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.945014 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.945168 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.947603 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:43.947606 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:43.947609 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:43.947646 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:43.949388 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.949943 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.949946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.949948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.950033 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.950360 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.950363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.950364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.950571 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:43.951490 (+ 919us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:43.951584 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 05:09:43.951768 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":190,"thread_start_us":98,"threads_started":1}
I20260807 05:09:43.952948 14507 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.16.124:44531: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d"
member_type: NON_VOTER
last_known_addr {
host: "127.8.16.124"
port: 44531
}
attrs {
promote: true
}
}
dest_uuid: "81f0ce27c75145b3ace095da2ddb85d4"
cas_config_opid_index: 5
I20260807 05:09:43.953841 14507 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: NON_VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: true } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: NON_VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: true } }
I20260807 05:09:43.955164 15092 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8b93668c0b5f46a8ad3b2c6c983d461d (127.8.16.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: NON_VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: true } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: NON_VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: true } } }
I20260807 05:09:43.955781 15236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:43.956745 15092 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.16.124:44531
I20260807 05:09:43.956943 15092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "81f0ce27c75145b3ace095da2ddb85d4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: NON_VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: true } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: NON_VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: true } } }
I20260807 05:09:43.957054 15092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:43.957285 15236 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [term 1 FOLLOWER]: Refusing update from remote peer 81f0ce27c75145b3ace095da2ddb85d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 05:09:43.957464 15104 master.cc:537] Master@127.8.16.124:44531 shutting down...
I20260807 05:09:43.957906 15089 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: NON_VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 05:09:43.959169 14492 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260807 05:09:43.962826 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.955028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:38779 (local address 127.0.0.1:46588)
0807 05:09:43.957352 (+ 2324us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:43.957366 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:43.957439 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:43.958805 (+ 1366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:43.958808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:43.958828 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:43.958917 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.958923 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.959965 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.959969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.961212 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.961225 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.962398 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.962401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.962499 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.962545 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 05:09:43.962618 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 05:09:43.962692 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2249}
I20260807 05:09:43.963357 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.955135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46588 (local address 127.8.16.125:38779)
0807 05:09:43.957674 (+ 2539us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:43.957678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:43.957693 (+ 15us) server_negotiation.cc:407] Connection header received
0807 05:09:43.957745 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:43.957747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:43.957813 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:43.957903 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:43.959023 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.959811 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.961351 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.962057 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.962171 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.962334 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:43.963153 (+ 819us) server_negotiation.cc:299] Negotiation successful
0807 05:09:43.963238 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2438}
I20260807 05:09:43.963804 15150 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 1 LEARNER]: Refusing update from remote peer 81f0ce27c75145b3ace095da2ddb85d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 05:09:43.964071 15104 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:43.964362 15104 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:43.964466 15104 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d: stopping tablet replica
I20260807 05:09:43.965059 15089 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: NON_VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 05:09:43.966223 15150 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8b93668c0b5f46a8ad3b2c6c983d461d (127.8.16.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: NON_VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: true } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: NON_VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: true } } }
I20260807 05:09:43.967937 15182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81f0ce27c75145b3ace095da2ddb85d4. Latest consensus state: current_term: 1 leader_uuid: "81f0ce27c75145b3ace095da2ddb85d4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: NON_VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: true } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: NON_VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: true } } }
I20260807 05:09:43.968098 15182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: This master's current role is: LEARNER
I20260807 05:09:43.968226 15182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "81f0ce27c75145b3ace095da2ddb85d4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: NON_VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: true } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: NON_VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: true } } }
I20260807 05:09:43.968317 15182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: This master's current role is: LEARNER
I20260807 05:09:43.968950 15090 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4: attempting to promote NON_VOTER 421e96d56e9f4523a75349dccff2cb5d to VOTER
I20260807 05:09:43.969390 15090 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: NON_VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: true } }
I20260807 05:09:43.970808 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.970497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:44531 (local address 127.0.0.1:47868)
0807 05:09:43.970649 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:43.970725 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:44531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 05:09:43.971197 15150 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 1 LEARNER]: Refusing update from remote peer 81f0ce27c75145b3ace095da2ddb85d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 05:09:43.971745 15092 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 05:09:43.972579 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.972318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:44531 (local address 127.0.0.1:47876)
0807 05:09:43.972462 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:43.972509 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:44531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 05:09:43.972697 15104 master.cc:552] Master@127.8.16.124:44531 shutdown complete.
I20260807 05:09:43.972754 15104 master_runner.cc:305] Clearing existing system tablet
W20260807 05:09:43.972779 14492 consensus_peers.cc:487] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 -> Peer 8b93668c0b5f46a8ad3b2c6c983d461d (127.8.16.124:44531): Couldn't send request to peer 8b93668c0b5f46a8ad3b2c6c983d461d. Status: Network error: Client connection negotiation failed: client connection to 127.8.16.124:44531: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 05:09:43.973002 15104 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:43.973055 15104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:43.974308 15104 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 05:09:43.974449 15092 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 421e96d56e9f4523a75349dccff2cb5d (127.8.16.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: NON_VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: true } } }
I20260807 05:09:43.976477 15150 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 421e96d56e9f4523a75349dccff2cb5d (127.8.16.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: NON_VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: true } } }
I20260807 05:09:43.977635 15298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:43.977824 15104 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:43.977927 15104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 05:09:43.977957 15104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/wal
uuid: "8b93668c0b5f46a8ad3b2c6c983d461d"
format_stamp: "Formatted at 2026-08-07 05:09:43 on dist-test-slave-49h3"
server_key: "ac29d53add3b42fc81b5304ea5c34bdd"
server_key_iv: "be2ff39f1d323d549ff2892925efa09c"
server_key_version: "encryptionkey@0"
I20260807 05:09:43.979202 15295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "81f0ce27c75145b3ace095da2ddb85d4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: NON_VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: true } } }
I20260807 05:09:43.979707 15090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "81f0ce27c75145b3ace095da2ddb85d4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: NON_VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: true } } }
I20260807 05:09:43.979822 15090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:43.979990 15104 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:43.980041 15090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "81f0ce27c75145b3ace095da2ddb85d4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: NON_VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: true } } }
I20260807 05:09:43.980131 15090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:43.980315 15295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:09:43.982334 15104 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 05:09:43.982427 15104 log.cc:1196] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 05:09:43.982686 15104 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d: Deleting consensus metadata
I20260807 05:09:43.982843 15104 master_runner.cc:315] Copying system tablet from 127.8.16.126:46471
I20260807 05:09:43.983999 15104 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d: tablet copy: Beginning tablet copy session from remote peer at address 127.8.16.126:46471
I20260807 05:09:43.993495 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.984568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39662 (local address 127.8.16.126:46471)
0807 05:09:43.984680 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:43.984685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:43.985127 (+ 442us) server_negotiation.cc:407] Connection header received
0807 05:09:43.985238 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:43.985241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:43.985292 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:43.985374 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:43.985940 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.986764 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.987904 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.988249 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.988346 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.989754 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:43.989758 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:43.989761 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:43.989790 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:43.991567 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.992103 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.992107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.992109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.992199 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:43.992491 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:43.992494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:43.992496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:43.992741 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:43.992833 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:43.993175 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 05:09:43.993295 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 05:09:43.993503 15304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:43.984768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.0.0.1:39662)
0807 05:09:43.985021 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:43.985037 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:43.985154 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:43.985405 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:43.985408 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:43.985689 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:43.985820 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.985828 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.986881 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.986884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.987786 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:43.987797 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:43.988363 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:43.988367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:43.988463 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:43.989059 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:43.989073 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:43.989591 (+ 518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:43.991693 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.991697 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.991700 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.991975 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.992298 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:43.992301 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:43.992303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:43.992407 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:43.992833 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:43.992836 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:43.992947 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 05:09:43.993055 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 05:09:43.993252 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":78,"threads_started":1}
I20260807 05:09:43.994306 14527 tablet_copy_service.cc:140] P 81f0ce27c75145b3ace095da2ddb85d4: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8b93668c0b5f46a8ad3b2c6c983d461d ({username='oryx', principal='oryx/127.8.16.124@KRBTEST.COM'} at 127.0.0.1:39662)
I20260807 05:09:43.994433 14527 tablet_copy_service.cc:161] P 81f0ce27c75145b3ace095da2ddb85d4: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8b93668c0b5f46a8ad3b2c6c983d461d at {username='oryx', principal='oryx/127.8.16.124@KRBTEST.COM'} at 127.0.0.1:39662: session id = 8b93668c0b5f46a8ad3b2c6c983d461d-00000000000000000000000000000000
I20260807 05:09:43.995513 14527 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 05:09:43.996810 15104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 05:09:43.999271 15104 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d: tablet copy: Starting download of 0 data blocks...
I20260807 05:09:43.999469 15104 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d: tablet copy: Starting download of 1 WAL segments...
I20260807 05:09:44.000979 15104 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 05:09:44.002974 14527 tablet_copy_service.cc:342] P 81f0ce27c75145b3ace095da2ddb85d4: Request end of tablet copy session 8b93668c0b5f46a8ad3b2c6c983d461d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.16.124@KRBTEST.COM'} at 127.0.0.1:39662
I20260807 05:09:44.003067 14527 tablet_copy_service.cc:433] P 81f0ce27c75145b3ace095da2ddb85d4: ending tablet copy session 8b93668c0b5f46a8ad3b2c6c983d461d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8b93668c0b5f46a8ad3b2c6c983d461d
I20260807 05:09:44.009908 15104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:44.011746 15305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:44.014245 15306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:44.014552 15308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:44.017855 15104 server_base.cc:1034] running on GCE node
I20260807 05:09:44.018039 15104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:44.018394 15104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:44.019538 15104 hybrid_clock.cc:648] HybridClock initialized: now 1786079384019524 us; error 32 us; skew 500 ppm
Aug 07 05:09:44 dist-test-slave-49h3 krb5kdc[14460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079384, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:44.021152 15104 init.cc:377] Logged in from keytab as oryx/127.8.16.124@KRBTEST.COM (short username oryx)
I20260807 05:09:44.021654 15104 webserver.cc:460] Webserver started at http://127.8.16.124:39967/ using document root <none> and password file <none>
I20260807 05:09:44.021832 15104 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:44.021873 15104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:44.023238 15104 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 05:09:44.024225 15315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:44.024487 15104 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 05:09:44.024570 15104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/wal
uuid: "8b93668c0b5f46a8ad3b2c6c983d461d"
format_stamp: "Formatted at 2026-08-07 05:09:43 on dist-test-slave-49h3"
server_key: "ac29d53add3b42fc81b5304ea5c34bdd"
server_key_iv: "be2ff39f1d323d549ff2892925efa09c"
server_key_version: "encryptionkey@0"
I20260807 05:09:44.024655 15104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:44.036404 15104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:44.038520 15104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:44.043251 15104 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.124:44531
I20260807 05:09:44.043358 15368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.124:44531 every 8 connection(s)
I20260807 05:09:44.043838 15104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/master-2/data/info.pb
I20260807 05:09:44.045800 15369 sys_catalog.cc:263] Verifying existing consensus state
I20260807 05:09:44.046986 15369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d: Bootstrap starting.
I20260807 05:09:44.058382 15369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:09:44.058676 15369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d: Bootstrap complete.
I20260807 05:09:44.059270 15369 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: NON_VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: true } }
I20260807 05:09:44.059420 15369 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8b93668c0b5f46a8ad3b2c6c983d461d, State: Initialized, Role: LEARNER
I20260807 05:09:44.059541 15369 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: NON_VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: true } }
I20260807 05:09:44.059909 15372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: NON_VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: true } } }
I20260807 05:09:44.060017 15369 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:44.060115 15372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [sys.catalog]: This master's current role is: LEARNER
I20260807 05:09:44.063637 15383 catalog_manager.cc:1259] Loaded cluster ID: 5f5f89a5f3fa42ecb10b2f77a3d886cc
I20260807 05:09:44.063768 15383 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d: loading cluster ID for follower catalog manager: success
I20260807 05:09:44.065945 15383 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d: acquiring CA information for follower catalog manager: success
I20260807 05:09:44.067332 15383 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 05:09:44.086166 15292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:44.086417 15292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:44.086460 15292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:44.090466 15292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:44.090526 15292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:44.090592 15292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:09:44.095914 15292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.65.keytab
--principal=oryx/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:35079
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=42179
--tserver_master_addrs=127.8.16.126:46471,127.8.16.125:38779,127.8.16.124:44531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:44.096922 15292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:44.097997 15292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:44.107138 15389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:44.107139 15391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:44.107334 15292 server_base.cc:1034] running on GCE node
W20260807 05:09:44.107159 15388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:44.107764 15292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:44.108316 15292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:44.109491 15292 hybrid_clock.cc:648] HybridClock initialized: now 1786079384109463 us; error 38 us; skew 500 ppm
Aug 07 05:09:44 dist-test-slave-49h3 krb5kdc[14460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079384, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:44.112106 15292 init.cc:377] Logged in from keytab as oryx/127.8.16.65@KRBTEST.COM (short username oryx)
I20260807 05:09:44.112937 15292 webserver.cc:460] Webserver started at http://127.8.16.65:42179/ using document root <none> and password file <none>
I20260807 05:09:44.113492 15292 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:44.113595 15292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:44.117264 15292 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:44.119164 15399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:44.120291 15292 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:44.120395 15292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "fcf7f715e0ba4d5690059e682961cf0c"
format_stamp: "Formatted at 2026-08-07 05:09:42 on dist-test-slave-49h3"
server_key: "dbeea2f34fc329bd9f3f900b66b5fc25"
server_key_iv: "fdc042aad1200e8a485b5259a364b18c"
server_key_version: "encryptionkey@0"
I20260807 05:09:44.120918 15292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:44.138065 15292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:44.141033 15292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:44.141211 15292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:44.141829 15292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:44.142650 15292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:44.142697 15292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:44.142747 15292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:44.142776 15292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:44.151388 15292 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:35079
I20260807 05:09:44.151444 15516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:35079 every 8 connection(s)
I20260807 05:09:44.152318 15292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:09:44.152505 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 15292
I20260807 05:09:44.152686 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 14714
I20260807 05:09:44.162802 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:37209
--local_ip_for_outbound_sockets=127.8.16.66
--tserver_master_addrs=127.8.16.126:46471
--webserver_port=38999
--webserver_interface=127.8.16.66
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.66.keytab
--principal=oryx/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.16.126:46471,127.8.16.125:38779
--tserver_master_addrs=127.8.16.126:46471,127.8.16.125:38779,127.8.16.124:44531 with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 05:09:44 dist-test-slave-49h3 krb5kdc[14460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079384, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM
Aug 07 05:09:44 dist-test-slave-49h3 krb5kdc[14460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079384, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.125@KRBTEST.COM
Aug 07 05:09:44 dist-test-slave-49h3 krb5kdc[14460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079384, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.124@KRBTEST.COM
I20260807 05:09:44.169571 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.155378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:59907 (local address 127.8.16.126:46471)
0807 05:09:44.155490 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:44.155494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:44.156318 (+ 824us) server_negotiation.cc:407] Connection header received
0807 05:09:44.156399 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:44.156402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:44.156454 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:44.156532 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:44.158096 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.158909 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.160334 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.160627 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.160698 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.165608 (+ 4910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:44.165612 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:44.165614 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:44.165641 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:44.167365 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.167890 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.167893 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.167895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.167989 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.168347 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.168350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.168352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.168572 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:44.168670 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:44.169287 (+ 617us) server_negotiation.cc:299] Negotiation successful
0807 05:09:44.169392 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 05:09:44.170135 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.155809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:57791 (local address 127.8.16.125:38779)
0807 05:09:44.155929 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:44.155933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:44.156617 (+ 684us) server_negotiation.cc:407] Connection header received
0807 05:09:44.156790 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:44.156794 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:44.156851 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:44.156903 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:44.157802 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.158544 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.160333 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.160672 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.160745 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.165572 (+ 4827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:44.165576 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:44.165579 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:44.165607 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:44.167220 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.167934 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.167938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.167940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.168019 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.168390 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.168393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.168395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.168584 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:44.168670 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:44.169866 (+ 1196us) server_negotiation.cc:299] Negotiation successful
0807 05:09:44.169983 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 05:09:44.170665 15522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.155391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:38779 (local address 127.8.16.65:57791)
0807 05:09:44.156527 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:44.156538 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:44.156682 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:44.157046 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:44.157049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:44.157183 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:44.157664 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.157679 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.158707 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.158710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.160194 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.160206 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.160914 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.160918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.161201 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.162045 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:44.162063 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:44.165432 (+ 3369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:44.167348 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:44.167354 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:44.167371 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:44.167795 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:44.168104 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:44.168107 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:44.168109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:44.168265 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:44.169524 (+ 1259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:44.169527 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:44.169623 (+ 96us) client_negotiation.cc:769] Sending connection context
0807 05:09:44.169796 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 05:09:44.169930 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1069,"mutex_wait_us":113,"spinlock_wait_cycles":4224,"thread_start_us":84,"threads_started":1}
I20260807 05:09:44.170905 15523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.154576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.8.16.65:59907)
0807 05:09:44.156211 (+ 1635us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:44.156223 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:44.156319 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:44.156665 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:44.156669 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:44.157081 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:44.157665 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.157681 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.159056 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.159059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.160196 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.160206 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.160895 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.160899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.161169 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.162101 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:44.162116 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:44.165158 (+ 3042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:44.167456 (+ 2298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:44.167459 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:44.167461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:44.167799 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:44.168095 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:44.168099 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:44.168101 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:44.168256 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:44.168678 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:44.168686 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:44.169022 (+ 336us) client_negotiation.cc:769] Sending connection context
0807 05:09:44.169240 (+ 218us) client_negotiation.cc:240] Negotiation successful
0807 05:09:44.169658 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1581,"mutex_wait_us":183,"spinlock_wait_cycles":30848,"thread_start_us":112,"threads_started":1}
I20260807 05:09:44.172004 15519 heartbeater.cc:347] Connected to a master server at 127.8.16.125:38779
I20260807 05:09:44.172243 15519 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:44.172652 15521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.154407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:44531 (local address 127.8.16.65:34305)
0807 05:09:44.154864 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:44.154913 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:44.155857 (+ 944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:44.156536 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:44.156546 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:44.157845 (+ 1299us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:44.157950 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.157956 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.159201 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.159205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.160226 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.160233 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.160897 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.160901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.161206 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.162019 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:44.162047 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:44.165067 (+ 3020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:44.171010 (+ 5943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:44.171013 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:44.171015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:44.171227 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:44.171543 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:44.171546 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:44.171548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:44.171651 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:44.172135 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:44.172139 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:44.172234 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 05:09:44.172331 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 05:09:44.172464 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":113,"spinlock_wait_cycles":4224,"thread_start_us":117,"threads_started":1}
I20260807 05:09:44.172711 15519 heartbeater.cc:511] Master 127.8.16.125:38779 requested a full tablet report, sending...
I20260807 05:09:44.173168 15524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.155754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:34305 (local address 127.8.16.124:44531)
0807 05:09:44.156117 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:44.156122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:44.156137 (+ 15us) server_negotiation.cc:407] Connection header received
0807 05:09:44.156183 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:44.156186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:44.156239 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:44.156961 (+ 722us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:44.158072 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.159074 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.160333 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.160704 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.160786 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.165261 (+ 4475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:44.165265 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:44.165268 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:44.165299 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:44.170880 (+ 5581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.171325 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.171329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.171331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.171431 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.171756 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.171760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.171761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.171993 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:44.172086 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:44.172868 (+ 782us) server_negotiation.cc:299] Negotiation successful
0807 05:09:44.173005 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":74,"threads_started":1}
I20260807 05:09:44.172004 15517 heartbeater.cc:347] Connected to a master server at 127.8.16.126:46471
I20260807 05:09:44.173261 15517 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:44.173409 15517 heartbeater.cc:511] Master 127.8.16.126:46471 requested a full tablet report, sending...
I20260807 05:09:44.174247 14507 ts_manager.cc:194] Re-registered known tserver with Master: fcf7f715e0ba4d5690059e682961cf0c (127.8.16.65:35079)
I20260807 05:09:44.174495 15139 ts_manager.cc:194] Registered new tserver with Master: fcf7f715e0ba4d5690059e682961cf0c (127.8.16.65:35079)
I20260807 05:09:44.175020 15518 heartbeater.cc:347] Connected to a master server at 127.8.16.124:44531
I20260807 05:09:44.175004 14507 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:59907
I20260807 05:09:44.175091 15518 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:44.175251 15518 heartbeater.cc:511] Master 127.8.16.124:44531 requested a full tablet report, sending...
I20260807 05:09:44.176221 15332 ts_manager.cc:194] Registered new tserver with Master: fcf7f715e0ba4d5690059e682961cf0c (127.8.16.65:35079)
W20260807 05:09:44.275902 15525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:44.276178 15525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:44.276240 15525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:44.280261 15525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:44.280326 15525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:44.280407 15525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:09:44.285530 15525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.66.keytab
--principal=oryx/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:37209
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=38999
--tserver_master_addrs=127.8.16.126:46471,127.8.16.125:38779,127.8.16.124:44531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:44.286540 15525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:44.287509 15525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:44.296943 15533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:44.296967 15530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:44.296967 15531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:44.297196 15525 server_base.cc:1034] running on GCE node
I20260807 05:09:44.297621 15525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:44.298190 15525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:44.299368 15525 hybrid_clock.cc:648] HybridClock initialized: now 1786079384299340 us; error 39 us; skew 500 ppm
Aug 07 05:09:44 dist-test-slave-49h3 krb5kdc[14460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079384, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:44.302091 15525 init.cc:377] Logged in from keytab as oryx/127.8.16.66@KRBTEST.COM (short username oryx)
I20260807 05:09:44.302891 15525 webserver.cc:460] Webserver started at http://127.8.16.66:38999/ using document root <none> and password file <none>
I20260807 05:09:44.303388 15525 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:44.303459 15525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:44.307114 15525 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:44.309023 15541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:44.310282 15525 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:44.310393 15525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "795c4878987947afa5d8ab2c58773e59"
format_stamp: "Formatted at 2026-08-07 05:09:43 on dist-test-slave-49h3"
server_key: "85c32836cc01e553dc395d99372a6185"
server_key_iv: "9cdbe591f330709dc68e56370a717435"
server_key_version: "encryptionkey@0"
I20260807 05:09:44.311000 15525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:44.327204 15525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:44.329973 15525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:44.330113 15525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:44.330610 15525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:44.331373 15525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:44.331418 15525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:44.331453 15525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:44.331473 15525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:44.339841 15525 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:37209
I20260807 05:09:44.339888 15659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:37209 every 8 connection(s)
I20260807 05:09:44.340803 15525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:09:44.350478 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 15525
I20260807 05:09:44.350690 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 14850
Aug 07 05:09:44 dist-test-slave-49h3 krb5kdc[14460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079384, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.124@KRBTEST.COM
Aug 07 05:09:44 dist-test-slave-49h3 krb5kdc[14460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079384, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.125@KRBTEST.COM
Aug 07 05:09:44 dist-test-slave-49h3 krb5kdc[14460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079384, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM
I20260807 05:09:44.358742 15524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.342700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:40217 (local address 127.8.16.124:44531)
0807 05:09:44.342928 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:44.342933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:44.344050 (+ 1117us) server_negotiation.cc:407] Connection header received
0807 05:09:44.345075 (+ 1025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:44.345078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:44.345136 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:44.345213 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:44.346927 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.347693 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.349653 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.350007 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.350094 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.353902 (+ 3808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:44.353906 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:44.353909 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:44.353940 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:44.355709 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.356329 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.356333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.356335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.356426 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.356989 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.356994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.356996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.357363 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:44.357514 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:44.358408 (+ 894us) server_negotiation.cc:299] Negotiation successful
0807 05:09:44.358555 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260807 05:09:44.359418 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.343322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:52327 (local address 127.8.16.125:38779)
0807 05:09:44.343448 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:44.343452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:44.344147 (+ 695us) server_negotiation.cc:407] Connection header received
0807 05:09:44.345055 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:44.345059 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:44.345128 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:44.345201 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:44.346886 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.347717 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.349675 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.350018 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.350094 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.353856 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:44.353859 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:44.353861 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:44.353888 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:44.356022 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.356663 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.356668 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.356670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.356773 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.358310 (+ 1537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.358314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.358315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.358599 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:44.358694 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:44.359093 (+ 399us) server_negotiation.cc:299] Negotiation successful
0807 05:09:44.359237 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 05:09:44.360628 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.342642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:41689 (local address 127.8.16.126:46471)
0807 05:09:44.342810 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:44.342814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:44.343839 (+ 1025us) server_negotiation.cc:407] Connection header received
0807 05:09:44.345215 (+ 1376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:44.345219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:44.345270 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:44.345346 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:44.346887 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.347717 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.349675 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.350026 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.350102 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.356700 (+ 6598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:44.356704 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:44.356706 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:44.356737 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:44.358362 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.358817 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.358821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.358822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.358910 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.359451 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.359454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.359456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.359685 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:44.359766 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:44.360347 (+ 581us) server_negotiation.cc:299] Negotiation successful
0807 05:09:44.360467 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 05:09:44.360652 15666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.343082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:44531 (local address 127.8.16.66:40217)
0807 05:09:44.343970 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:44.343981 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:44.344857 (+ 876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:44.345397 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:44.345406 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:44.346003 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:44.346715 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.346748 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.347898 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.347901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.349522 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.349532 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.350288 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.350292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.350583 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.351493 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:44.351521 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:44.353663 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:44.355856 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:44.355863 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:44.355879 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:44.356212 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:44.356532 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:44.356534 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:44.356537 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:44.356708 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:44.357613 (+ 905us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:44.357620 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:44.358122 (+ 502us) client_negotiation.cc:769] Sending connection context
0807 05:09:44.358985 (+ 863us) client_negotiation.cc:240] Negotiation successful
0807 05:09:44.359353 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"mutex_wait_us":334,"spinlock_wait_cycles":22656,"thread_start_us":128,"threads_started":1}
I20260807 05:09:44.360656 15664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.343082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:38779 (local address 127.8.16.66:52327)
0807 05:09:44.344059 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:44.344070 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:44.344849 (+ 779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:44.345396 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:44.345405 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:44.346003 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:44.346717 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.346737 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.347899 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.347902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.349535 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.349546 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.350288 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.350291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.350593 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.351501 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:44.351517 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:44.353663 (+ 2146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:44.356164 (+ 2501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:44.356168 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:44.356171 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:44.356499 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:44.358059 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:44.358062 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:44.358064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:44.358184 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:44.358760 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:44.358764 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:44.358874 (+ 110us) client_negotiation.cc:769] Sending connection context
0807 05:09:44.359576 (+ 702us) client_negotiation.cc:240] Negotiation successful
0807 05:09:44.359753 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":903,"mutex_wait_us":494,"spinlock_wait_cycles":69376,"thread_start_us":119,"threads_started":1}
I20260807 05:09:44.361621 15665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.343081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.8.16.66:41689)
0807 05:09:44.343688 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:44.343739 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:44.345110 (+ 1371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:44.345396 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:44.345405 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:44.346005 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:44.346717 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.346737 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.347898 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.347902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.349535 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.349546 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.354405 (+ 4859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.354408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.354521 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.355037 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:44.355053 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:44.356532 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:44.358480 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:44.358482 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:44.358484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:44.358699 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:44.359252 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:44.359255 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:44.359257 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:44.359345 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:44.360041 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:44.360043 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:44.360132 (+ 89us) client_negotiation.cc:769] Sending connection context
0807 05:09:44.361329 (+ 1197us) client_negotiation.cc:240] Negotiation successful
0807 05:09:44.361493 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":322,"spinlock_wait_cycles":25216,"thread_start_us":133,"threads_started":1}
I20260807 05:09:44.362584 15661 heartbeater.cc:347] Connected to a master server at 127.8.16.124:44531
I20260807 05:09:44.362602 15660 heartbeater.cc:347] Connected to a master server at 127.8.16.126:46471
I20260807 05:09:44.362828 15661 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:44.362828 15660 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:44.363374 15660 heartbeater.cc:511] Master 127.8.16.126:46471 requested a full tablet report, sending...
I20260807 05:09:44.363373 15661 heartbeater.cc:511] Master 127.8.16.124:44531 requested a full tablet report, sending...
I20260807 05:09:44.364425 14507 ts_manager.cc:194] Re-registered known tserver with Master: 795c4878987947afa5d8ab2c58773e59 (127.8.16.66:37209)
I20260807 05:09:44.364657 15332 ts_manager.cc:194] Registered new tserver with Master: 795c4878987947afa5d8ab2c58773e59 (127.8.16.66:37209)
I20260807 05:09:44.365142 14507 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:41689
I20260807 05:09:44.366043 15662 heartbeater.cc:347] Connected to a master server at 127.8.16.125:38779
I20260807 05:09:44.366164 15662 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:44.366425 15662 heartbeater.cc:511] Master 127.8.16.125:38779 requested a full tablet report, sending...
I20260807 05:09:44.367101 15139 ts_manager.cc:194] Registered new tserver with Master: 795c4878987947afa5d8ab2c58773e59 (127.8.16.66:37209)
I20260807 05:09:44.368112 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:38737
--local_ip_for_outbound_sockets=127.8.16.67
--tserver_master_addrs=127.8.16.126:46471
--webserver_port=45311
--webserver_interface=127.8.16.67
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.67.keytab
--principal=oryx/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.16.126:46471,127.8.16.125:38779
--tserver_master_addrs=127.8.16.126:46471,127.8.16.125:38779,127.8.16.124:44531 with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:09:44.384030 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.380367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:44531 (local address 127.0.0.1:47886)
0807 05:09:44.380584 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:44.380597 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:44.380690 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:44.380959 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:44.380961 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:44.380972 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:44.381054 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.381058 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.382122 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.382124 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.382844 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.382852 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.383741 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.383744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.383806 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.383824 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 05:09:44.383865 (+ 41us) client_negotiation.cc:240] Negotiation successful
0807 05:09:44.383921 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260807 05:09:44.384121 15524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.380459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47886 (local address 127.8.16.124:44531)
0807 05:09:44.380587 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:44.380592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:44.380644 (+ 52us) server_negotiation.cc:407] Connection header received
0807 05:09:44.380784 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:44.380787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:44.380849 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:44.380943 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:44.381145 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.382001 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.382966 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.383637 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.383751 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.383908 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:44.383942 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 05:09:44.384003 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 05:09:44.384727 15335 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 05:09:44.385154 15372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81f0ce27c75145b3ace095da2ddb85d4. Latest consensus state: current_term: 1 leader_uuid: "81f0ce27c75145b3ace095da2ddb85d4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: NON_VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: true } } }
I20260807 05:09:44.385293 15372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [sys.catalog]: This master's current role is: LEARNER
W20260807 05:09:44.477480 15667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:44.477741 15667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:44.477788 15667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:44.481769 15667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:44.481827 15667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:44.481894 15667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:09:44.487003 15667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.67.keytab
--principal=oryx/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:38737
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=45311
--tserver_master_addrs=127.8.16.126:46471,127.8.16.125:38779,127.8.16.124:44531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:44.487970 15667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:44.488945 15667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:44.498138 15673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:44.498123 15674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:44.498109 15676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:44.498322 15667 server_base.cc:1034] running on GCE node
I20260807 05:09:44.498782 15667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:44.499388 15667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:44.500553 15667 hybrid_clock.cc:648] HybridClock initialized: now 1786079384500539 us; error 28 us; skew 500 ppm
Aug 07 05:09:44 dist-test-slave-49h3 krb5kdc[14460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079384, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:44.503255 15667 init.cc:377] Logged in from keytab as oryx/127.8.16.67@KRBTEST.COM (short username oryx)
I20260807 05:09:44.504123 15667 webserver.cc:460] Webserver started at http://127.8.16.67:45311/ using document root <none> and password file <none>
I20260807 05:09:44.504626 15667 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:44.504707 15667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:44.508316 15667 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:44.510258 15684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:44.511402 15667 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:44.511510 15667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "aebc96dd8d22409caba2a0968488a3b8"
format_stamp: "Formatted at 2026-08-07 05:09:43 on dist-test-slave-49h3"
server_key: "1e684f7f3243ef94441a24190a305759"
server_key_iv: "dd732459123d3ea93e8c2468c094a481"
server_key_version: "encryptionkey@0"
I20260807 05:09:44.512018 15667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:44.567332 15667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:44.570106 15667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:44.570281 15667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:44.570861 15667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:44.571738 15667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:44.571787 15667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:44.571839 15667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:44.571866 15667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:44.580304 15667 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:38737
I20260807 05:09:44.580341 15801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:38737 every 8 connection(s)
I20260807 05:09:44.581274 15667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:09:44.584790 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 15667
Aug 07 05:09:44 dist-test-slave-49h3 krb5kdc[14460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079384, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.124@KRBTEST.COM
Aug 07 05:09:44 dist-test-slave-49h3 krb5kdc[14460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079384, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.125@KRBTEST.COM
Aug 07 05:09:44 dist-test-slave-49h3 krb5kdc[14460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079384, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM
I20260807 05:09:44.596818 15818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.587245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46590 (local address 127.8.16.125:38779)
0807 05:09:44.587742 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:44.587746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:44.587900 (+ 154us) server_negotiation.cc:407] Connection header received
0807 05:09:44.587940 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:44.587943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:44.587985 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:44.588053 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:44.588876 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.589690 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.590785 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.591105 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.591192 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.592923 (+ 1731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:44.592927 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:44.592930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:44.592955 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:44.594790 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.595278 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.595282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.595284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.595366 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.595652 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.595655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.595657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.595861 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:44.595972 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:44.596539 (+ 567us) server_negotiation.cc:299] Negotiation successful
0807 05:09:44.596650 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":57,"threads_started":1}
I20260807 05:09:44.600730 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.583620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:45687 (local address 127.8.16.125:38779)
0807 05:09:44.583732 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:44.583737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:44.584295 (+ 558us) server_negotiation.cc:407] Connection header received
0807 05:09:44.585527 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:44.585531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:44.585613 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:44.585688 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:44.587539 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.588276 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.590045 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.590374 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.590460 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.595926 (+ 5466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:44.595929 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:44.595931 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:44.595951 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:44.598348 (+ 2397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.598859 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.598862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.598864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.598959 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.599363 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.599366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.599368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.599558 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:44.599644 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:44.600501 (+ 857us) server_negotiation.cc:299] Negotiation successful
0807 05:09:44.600598 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 05:09:44.601131 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.583617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:40365 (local address 127.8.16.126:46471)
0807 05:09:44.583744 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:44.583748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:44.584859 (+ 1111us) server_negotiation.cc:407] Connection header received
0807 05:09:44.585551 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:44.585554 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:44.585611 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:44.585665 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:44.588966 (+ 3301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.589773 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.593873 (+ 4100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.594206 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.594309 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.596665 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:44.596668 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:44.596670 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:44.596695 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:44.598224 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.598859 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.598862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.598864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.598960 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.599363 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.599366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.599368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.599558 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:44.599645 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:44.600933 (+ 1288us) server_negotiation.cc:299] Negotiation successful
0807 05:09:44.601023 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:09:44.601161 15815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.586808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47902 (local address 127.8.16.124:44531)
0807 05:09:44.587397 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:44.587401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:44.587701 (+ 300us) server_negotiation.cc:407] Connection header received
0807 05:09:44.587755 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:44.587758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:44.587805 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:44.587882 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:44.588714 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.591329 (+ 2615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.592361 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.592719 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.592899 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.594530 (+ 1631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:44.594533 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:44.594536 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:44.594560 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:44.599153 (+ 4593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.599612 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.599615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.599617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.599697 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.599976 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.599979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.599980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.600184 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:44.600312 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:44.600880 (+ 568us) server_negotiation.cc:299] Negotiation successful
0807 05:09:44.600993 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":84,"threads_started":1}
I20260807 05:09:44.602576 15808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.583419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:38779 (local address 127.8.16.67:45687)
0807 05:09:44.584146 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:44.584196 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:44.585214 (+ 1018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:44.586079 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:44.586088 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:44.586694 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:44.587379 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.587398 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.588434 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.588438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.589920 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.589931 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.590594 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.590598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.592995 (+ 2397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.593805 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:44.593820 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:44.595796 (+ 1976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:44.598463 (+ 2667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:44.598466 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:44.598468 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:44.598756 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:44.599059 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:44.599062 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:44.599064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:44.599256 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:44.599699 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:44.599706 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:44.600046 (+ 340us) client_negotiation.cc:769] Sending connection context
0807 05:09:44.600249 (+ 203us) client_negotiation.cc:240] Negotiation successful
0807 05:09:44.601712 (+ 1463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"mutex_wait_us":3551,"spinlock_wait_cycles":19456,"thread_start_us":139,"threads_started":1}
I20260807 05:09:44.602845 15806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.583430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.8.16.67:40365)
0807 05:09:44.584771 (+ 1341us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:44.584783 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:44.585406 (+ 623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:44.586079 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:44.586088 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:44.586699 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:44.588841 (+ 2142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.588848 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.589911 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.589914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.593744 (+ 3830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.593755 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.594425 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.594429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.594522 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.595125 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:44.595139 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:44.596520 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:44.598329 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:44.598335 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:44.598349 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:44.598746 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:44.599059 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:44.599062 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:44.599064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:44.599256 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:44.599699 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:44.599706 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:44.600053 (+ 347us) client_negotiation.cc:769] Sending connection context
0807 05:09:44.600249 (+ 196us) client_negotiation.cc:240] Negotiation successful
0807 05:09:44.600589 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1266,"mutex_wait_us":3375,"spinlock_wait_cycles":17664,"thread_start_us":115,"threads_started":1}
I20260807 05:09:44.603737 15524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.583008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:59341 (local address 127.8.16.124:44531)
0807 05:09:44.583143 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:44.583148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:44.584499 (+ 1351us) server_negotiation.cc:407] Connection header received
0807 05:09:44.586309 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:44.586313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:44.586369 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:44.586420 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:44.587996 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.588775 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.590019 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.590348 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.590437 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.596232 (+ 5795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:44.596236 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:44.596238 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:44.596264 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:44.599853 (+ 3589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.601854 (+ 2001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.601858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.601859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.601939 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.602270 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.602273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.602275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.602486 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:44.602596 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:44.603474 (+ 878us) server_negotiation.cc:299] Negotiation successful
0807 05:09:44.603600 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":57}
I20260807 05:09:44.603856 15814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.586716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39670 (local address 127.8.16.126:46471)
0807 05:09:44.590800 (+ 4084us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:44.590812 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:44.590826 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:09:44.590860 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:44.590862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:44.590905 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:44.591750 (+ 845us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:44.591846 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.592729 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.596693 (+ 3964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.597045 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.597472 (+ 427us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.598616 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:44.598619 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:44.598622 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:44.598645 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:44.601382 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.602257 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.602260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.602262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.602339 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.602635 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.602638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.602639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.602850 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:44.603515 (+ 665us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:44.603585 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 05:09:44.603696 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4009,"thread_start_us":91,"threads_started":1}
I20260807 05:09:44.603868 15802 heartbeater.cc:347] Connected to a master server at 127.8.16.126:46471
I20260807 05:09:44.603930 15804 heartbeater.cc:347] Connected to a master server at 127.8.16.125:38779
I20260807 05:09:44.604074 15802 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:44.604079 15804 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:44.604507 15807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.583420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:44531 (local address 127.8.16.67:59341)
0807 05:09:44.584409 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:44.584420 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:44.585214 (+ 794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:44.586569 (+ 1355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:44.586572 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:44.586863 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:44.587883 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.587890 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.588877 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.588880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.589912 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.589919 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.590584 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.590588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.590808 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.591441 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:44.591461 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:44.594418 (+ 2957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:44.601281 (+ 6863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:44.601284 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:44.601286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:44.601537 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:44.602040 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:44.602043 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:44.602045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:44.602128 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:44.602863 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:44.602866 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:44.603287 (+ 421us) client_negotiation.cc:769] Sending connection context
0807 05:09:44.604194 (+ 907us) client_negotiation.cc:240] Negotiation successful
0807 05:09:44.604342 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"mutex_wait_us":929,"spinlock_wait_cycles":30720,"thread_start_us":107,"threads_started":1}
I20260807 05:09:44.604755 15802 heartbeater.cc:511] Master 127.8.16.126:46471 requested a full tablet report, sending...
I20260807 05:09:44.604667 15804 heartbeater.cc:511] Master 127.8.16.125:38779 requested a full tablet report, sending...
I20260807 05:09:44.605489 15803 heartbeater.cc:347] Connected to a master server at 127.8.16.124:44531
I20260807 05:09:44.605572 15803 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:44.605721 15803 heartbeater.cc:511] Master 127.8.16.124:44531 requested a full tablet report, sending...
I20260807 05:09:44.605841 14507 ts_manager.cc:194] Re-registered known tserver with Master: aebc96dd8d22409caba2a0968488a3b8 (127.8.16.67:38737)
I20260807 05:09:44.605861 15139 ts_manager.cc:194] Registered new tserver with Master: aebc96dd8d22409caba2a0968488a3b8 (127.8.16.67:38737)
I20260807 05:09:44.606487 14507 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:40365
I20260807 05:09:44.606952 15332 ts_manager.cc:194] Registered new tserver with Master: aebc96dd8d22409caba2a0968488a3b8 (127.8.16.67:38737)
I20260807 05:09:44.606900 14506 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:09:44.609823 14506 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:09:44.612046 15092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "81f0ce27c75145b3ace095da2ddb85d4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: NON_VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: true } } }
I20260807 05:09:44.612170 15092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:44.615391 15092 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4: attempting to promote NON_VOTER 8b93668c0b5f46a8ad3b2c6c983d461d to VOTER
I20260807 05:09:44.615717 15092 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: false } }
I20260807 05:09:44.616809 15150 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 1 FOLLOWER]: Refusing update from remote peer 81f0ce27c75145b3ace095da2ddb85d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 05:09:44.617189 15824 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 05:09:44.617579 15336 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [term 1 LEARNER]: Refusing update from remote peer 81f0ce27c75145b3ace095da2ddb85d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 05:09:44.618201 15822 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 05:09:44.618709 15823 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 8b93668c0b5f46a8ad3b2c6c983d461d (127.8.16.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: false } } }
I20260807 05:09:44.619189 15150 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 8b93668c0b5f46a8ad3b2c6c983d461d (127.8.16.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: false } } }
I20260807 05:09:44.620107 15823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "81f0ce27c75145b3ace095da2ddb85d4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: false } } }
I20260807 05:09:44.620209 15823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:44.620283 15824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "81f0ce27c75145b3ace095da2ddb85d4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: false } } }
I20260807 05:09:44.620368 15824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:44.620404 15823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "81f0ce27c75145b3ace095da2ddb85d4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: false } } }
I20260807 05:09:44.620476 15823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:44.620884 15336 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 8b93668c0b5f46a8ad3b2c6c983d461d (127.8.16.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: false } } }
I20260807 05:09:44.622507 15372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "81f0ce27c75145b3ace095da2ddb85d4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: false } } }
I20260807 05:09:44.622611 15372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:09:44.624862 15832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "81f0ce27c75145b3ace095da2ddb85d4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: false } } }
I20260807 05:09:44.624989 15832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:09:44.629200 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.623588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:35079 (local address 127.0.0.1:46812)
0807 05:09:44.623717 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:44.623731 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:44.623808 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:44.625391 (+ 1583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:44.625394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:44.625434 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:44.625547 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.625553 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.626638 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.626641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.627805 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.627816 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.628825 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.628828 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.628909 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.628930 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 05:09:44.629003 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 05:09:44.629073 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":41}
I20260807 05:09:44.629256 15834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.624108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46812 (local address 127.8.16.65:35079)
0807 05:09:44.624606 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:44.624622 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:44.624647 (+ 25us) server_negotiation.cc:407] Connection header received
0807 05:09:44.624726 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:44.624732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:44.624917 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:44.625441 (+ 524us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:44.625678 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.626553 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.627926 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.628721 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.628812 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.628957 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:44.629035 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 05:09:44.629129 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":73,"threads_started":1}
I20260807 05:09:44.629418 15833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.623880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:38737 (local address 127.0.0.1:56280)
0807 05:09:44.624086 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:44.624099 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:44.624181 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:44.625243 (+ 1062us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:44.625246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:44.625259 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:44.625361 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.625367 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.626714 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.626717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.628056 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.628063 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.629041 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.629045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.629129 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.629151 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 05:09:44.629219 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 05:09:44.629286 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":21,"thread_start_us":85,"threads_started":1}
I20260807 05:09:44.629868 15836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.624466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56280 (local address 127.8.16.67:38737)
0807 05:09:44.624760 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:44.624772 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:44.624794 (+ 22us) server_negotiation.cc:407] Connection header received
0807 05:09:44.624862 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:44.624867 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:44.625122 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:44.625255 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:44.625521 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.626626 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.628156 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.628750 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.628852 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.628980 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:44.629604 (+ 624us) server_negotiation.cc:299] Negotiation successful
0807 05:09:44.629702 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":96,"threads_started":1}
I20260807 05:09:44.630014 15835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.624122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:37209 (local address 127.0.0.1:35204)
0807 05:09:44.625164 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:44.625174 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:44.625246 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:44.626194 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:44.626197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:44.626215 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:44.626305 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.626311 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.627423 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.627427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.628461 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.628472 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.629687 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.629690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.629772 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.629794 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 05:09:44.629841 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 05:09:44.629903 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":985,"mutex_wait_us":3,"thread_start_us":87,"threads_started":1}
I20260807 05:09:44.630594 15837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.624498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35204 (local address 127.8.16.66:37209)
0807 05:09:44.625741 (+ 1243us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:44.625751 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:44.625772 (+ 21us) server_negotiation.cc:407] Connection header received
0807 05:09:44.625840 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:44.625844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:44.626004 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:44.626186 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:44.626401 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.627328 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.628588 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.629485 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.630181 (+ 696us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.630324 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:44.630370 (+ 46us) server_negotiation.cc:299] Negotiation successful
0807 05:09:44.630448 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1146,"thread_start_us":126,"threads_started":1}
I20260807 05:09:44.632182 15448 tablet_service.cc:1467] Processing CreateTablet for tablet fa7601a7547a41ae85e5fadd135a00bd (DEFAULT_TABLE table=test-table [id=366eb0e5ab264928b35b270f342197f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:44.632197 15735 tablet_service.cc:1467] Processing CreateTablet for tablet fa7601a7547a41ae85e5fadd135a00bd (DEFAULT_TABLE table=test-table [id=366eb0e5ab264928b35b270f342197f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:44.633283 15735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa7601a7547a41ae85e5fadd135a00bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:44.633275 15448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa7601a7547a41ae85e5fadd135a00bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:44.639633 15590 tablet_service.cc:1467] Processing CreateTablet for tablet fa7601a7547a41ae85e5fadd135a00bd (DEFAULT_TABLE table=test-table [id=366eb0e5ab264928b35b270f342197f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:44.640445 15590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa7601a7547a41ae85e5fadd135a00bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:44.641201 15839 tablet_bootstrap.cc:492] T fa7601a7547a41ae85e5fadd135a00bd P aebc96dd8d22409caba2a0968488a3b8: Bootstrap starting.
I20260807 05:09:44.641618 15838 tablet_bootstrap.cc:492] T fa7601a7547a41ae85e5fadd135a00bd P fcf7f715e0ba4d5690059e682961cf0c: Bootstrap starting.
I20260807 05:09:44.643841 15838 tablet_bootstrap.cc:654] T fa7601a7547a41ae85e5fadd135a00bd P fcf7f715e0ba4d5690059e682961cf0c: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:44.644671 15838 log.cc:824] T fa7601a7547a41ae85e5fadd135a00bd P fcf7f715e0ba4d5690059e682961cf0c: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:44.646657 15838 tablet_bootstrap.cc:492] T fa7601a7547a41ae85e5fadd135a00bd P fcf7f715e0ba4d5690059e682961cf0c: No bootstrap required, opened a new log
I20260807 05:09:44.646838 15838 ts_tablet_manager.cc:1397] T fa7601a7547a41ae85e5fadd135a00bd P fcf7f715e0ba4d5690059e682961cf0c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:09:44.647221 15839 tablet_bootstrap.cc:654] T fa7601a7547a41ae85e5fadd135a00bd P aebc96dd8d22409caba2a0968488a3b8: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:44.648053 15839 log.cc:824] T fa7601a7547a41ae85e5fadd135a00bd P aebc96dd8d22409caba2a0968488a3b8: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:44.648689 15841 tablet_bootstrap.cc:492] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59: Bootstrap starting.
I20260807 05:09:44.650008 15839 tablet_bootstrap.cc:492] T fa7601a7547a41ae85e5fadd135a00bd P aebc96dd8d22409caba2a0968488a3b8: No bootstrap required, opened a new log
I20260807 05:09:44.650192 15839 ts_tablet_manager.cc:1397] T fa7601a7547a41ae85e5fadd135a00bd P aebc96dd8d22409caba2a0968488a3b8: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260807 05:09:44.650687 15841 tablet_bootstrap.cc:654] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:44.650729 15838 raft_consensus.cc:348] T fa7601a7547a41ae85e5fadd135a00bd P fcf7f715e0ba4d5690059e682961cf0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7f715e0ba4d5690059e682961cf0c" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 35079 } } peers { permanent_uuid: "aebc96dd8d22409caba2a0968488a3b8" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38737 } } peers { permanent_uuid: "795c4878987947afa5d8ab2c58773e59" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37209 } }
I20260807 05:09:44.650959 15838 raft_consensus.cc:374] T fa7601a7547a41ae85e5fadd135a00bd P fcf7f715e0ba4d5690059e682961cf0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:44.651010 15838 raft_consensus.cc:729] T fa7601a7547a41ae85e5fadd135a00bd P fcf7f715e0ba4d5690059e682961cf0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf7f715e0ba4d5690059e682961cf0c, State: Initialized, Role: FOLLOWER
I20260807 05:09:44.651399 15838 consensus_queue.cc:260] T fa7601a7547a41ae85e5fadd135a00bd P fcf7f715e0ba4d5690059e682961cf0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7f715e0ba4d5690059e682961cf0c" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 35079 } } peers { permanent_uuid: "aebc96dd8d22409caba2a0968488a3b8" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38737 } } peers { permanent_uuid: "795c4878987947afa5d8ab2c58773e59" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37209 } }
I20260807 05:09:44.651445 15841 log.cc:824] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:44.651986 15517 heartbeater.cc:503] Master 127.8.16.126:46471 was elected leader, sending a full tablet report...
I20260807 05:09:44.652323 15838 ts_tablet_manager.cc:1428] T fa7601a7547a41ae85e5fadd135a00bd P fcf7f715e0ba4d5690059e682961cf0c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 05:09:44.653301 15841 tablet_bootstrap.cc:492] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59: No bootstrap required, opened a new log
I20260807 05:09:44.653509 15841 ts_tablet_manager.cc:1397] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
W20260807 05:09:44.654441 15520 tablet.cc:2369] T fa7601a7547a41ae85e5fadd135a00bd P fcf7f715e0ba4d5690059e682961cf0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:09:44.654546 15839 raft_consensus.cc:348] T fa7601a7547a41ae85e5fadd135a00bd P aebc96dd8d22409caba2a0968488a3b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7f715e0ba4d5690059e682961cf0c" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 35079 } } peers { permanent_uuid: "aebc96dd8d22409caba2a0968488a3b8" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38737 } } peers { permanent_uuid: "795c4878987947afa5d8ab2c58773e59" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37209 } }
I20260807 05:09:44.654808 15839 raft_consensus.cc:374] T fa7601a7547a41ae85e5fadd135a00bd P aebc96dd8d22409caba2a0968488a3b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:44.654853 15839 raft_consensus.cc:729] T fa7601a7547a41ae85e5fadd135a00bd P aebc96dd8d22409caba2a0968488a3b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aebc96dd8d22409caba2a0968488a3b8, State: Initialized, Role: FOLLOWER
I20260807 05:09:44.657123 15839 consensus_queue.cc:260] T fa7601a7547a41ae85e5fadd135a00bd P aebc96dd8d22409caba2a0968488a3b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7f715e0ba4d5690059e682961cf0c" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 35079 } } peers { permanent_uuid: "aebc96dd8d22409caba2a0968488a3b8" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38737 } } peers { permanent_uuid: "795c4878987947afa5d8ab2c58773e59" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37209 } }
I20260807 05:09:44.657699 15841 raft_consensus.cc:348] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7f715e0ba4d5690059e682961cf0c" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 35079 } } peers { permanent_uuid: "aebc96dd8d22409caba2a0968488a3b8" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38737 } } peers { permanent_uuid: "795c4878987947afa5d8ab2c58773e59" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37209 } }
I20260807 05:09:44.657940 15841 raft_consensus.cc:374] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:44.657995 15841 raft_consensus.cc:729] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 795c4878987947afa5d8ab2c58773e59, State: Initialized, Role: FOLLOWER
I20260807 05:09:44.658203 15802 heartbeater.cc:503] Master 127.8.16.126:46471 was elected leader, sending a full tablet report...
I20260807 05:09:44.658387 15841 consensus_queue.cc:260] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7f715e0ba4d5690059e682961cf0c" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 35079 } } peers { permanent_uuid: "aebc96dd8d22409caba2a0968488a3b8" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38737 } } peers { permanent_uuid: "795c4878987947afa5d8ab2c58773e59" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37209 } }
I20260807 05:09:44.658627 15839 ts_tablet_manager.cc:1428] T fa7601a7547a41ae85e5fadd135a00bd P aebc96dd8d22409caba2a0968488a3b8: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 05:09:44.659263 15841 ts_tablet_manager.cc:1428] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 05:09:44.659323 15660 heartbeater.cc:503] Master 127.8.16.126:46471 was elected leader, sending a full tablet report...
I20260807 05:09:44.755468 15846 raft_consensus.cc:482] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:09:44.755677 15846 raft_consensus.cc:504] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7f715e0ba4d5690059e682961cf0c" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 35079 } } peers { permanent_uuid: "aebc96dd8d22409caba2a0968488a3b8" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38737 } } peers { permanent_uuid: "795c4878987947afa5d8ab2c58773e59" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37209 } }
I20260807 05:09:44.756858 15846 leader_election.cc:288] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcf7f715e0ba4d5690059e682961cf0c (127.8.16.65:35079), aebc96dd8d22409caba2a0968488a3b8 (127.8.16.67:38737)
I20260807 05:09:44.760456 15834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.757251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:51463 (local address 127.8.16.65:35079)
0807 05:09:44.757370 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:44.757374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:44.757393 (+ 19us) server_negotiation.cc:407] Connection header received
0807 05:09:44.757544 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:44.757547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:44.757602 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:44.757677 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:44.757951 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.758455 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.759508 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.759944 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.760037 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.760103 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:44.760272 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 05:09:44.760341 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 05:09:44.760457 15665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.757149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:35079 (local address 127.8.16.66:51463)
0807 05:09:44.757287 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:44.757302 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:44.757395 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:44.757695 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:44.757698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:44.757717 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:44.757842 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.757849 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.758562 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.758565 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.759359 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.759369 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.760050 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.760053 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.760156 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.760176 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 05:09:44.760242 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 05:09:44.760317 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 05:09:44.760608 15836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.757567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:50733 (local address 127.8.16.67:38737)
0807 05:09:44.757711 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:44.757716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:44.757729 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:09:44.757776 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:44.757778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:44.757819 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:44.757879 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:44.758145 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.758652 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.759642 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.760039 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.760132 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.760191 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:44.760386 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 05:09:44.760457 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 05:09:44.760733 15664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.757421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:38737 (local address 127.8.16.66:50733)
0807 05:09:44.757574 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:44.757585 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:44.757650 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:44.757901 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:44.757904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:44.757918 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:44.758024 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.758029 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.758757 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.758761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.759526 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:44.759536 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.760147 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.760151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:44.760276 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.760296 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 05:09:44.760358 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 05:09:44.760427 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":1}
I20260807 05:09:44.761390 15755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa7601a7547a41ae85e5fadd135a00bd" candidate_uuid: "795c4878987947afa5d8ab2c58773e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aebc96dd8d22409caba2a0968488a3b8" is_pre_election: true
I20260807 05:09:44.761382 15469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa7601a7547a41ae85e5fadd135a00bd" candidate_uuid: "795c4878987947afa5d8ab2c58773e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7f715e0ba4d5690059e682961cf0c" is_pre_election: true
I20260807 05:09:44.761691 15755 raft_consensus.cc:2454] T fa7601a7547a41ae85e5fadd135a00bd P aebc96dd8d22409caba2a0968488a3b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 795c4878987947afa5d8ab2c58773e59 in term 0.
I20260807 05:09:44.761703 15469 raft_consensus.cc:2454] T fa7601a7547a41ae85e5fadd135a00bd P fcf7f715e0ba4d5690059e682961cf0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 795c4878987947afa5d8ab2c58773e59 in term 0.
I20260807 05:09:44.762151 15542 leader_election.cc:302] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 795c4878987947afa5d8ab2c58773e59, fcf7f715e0ba4d5690059e682961cf0c; no voters:
I20260807 05:09:44.762425 15846 raft_consensus.cc:2781] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:09:44.762509 15846 raft_consensus.cc:482] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:09:44.762560 15846 raft_consensus.cc:3037] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:44.763602 15846 raft_consensus.cc:504] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7f715e0ba4d5690059e682961cf0c" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 35079 } } peers { permanent_uuid: "aebc96dd8d22409caba2a0968488a3b8" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38737 } } peers { permanent_uuid: "795c4878987947afa5d8ab2c58773e59" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37209 } }
I20260807 05:09:44.763926 15846 leader_election.cc:288] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59 [CANDIDATE]: Term 1 election: Requested vote from peers fcf7f715e0ba4d5690059e682961cf0c (127.8.16.65:35079), aebc96dd8d22409caba2a0968488a3b8 (127.8.16.67:38737)
I20260807 05:09:44.764320 15469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa7601a7547a41ae85e5fadd135a00bd" candidate_uuid: "795c4878987947afa5d8ab2c58773e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7f715e0ba4d5690059e682961cf0c"
I20260807 05:09:44.764380 15755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa7601a7547a41ae85e5fadd135a00bd" candidate_uuid: "795c4878987947afa5d8ab2c58773e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aebc96dd8d22409caba2a0968488a3b8"
I20260807 05:09:44.764452 15469 raft_consensus.cc:3037] T fa7601a7547a41ae85e5fadd135a00bd P fcf7f715e0ba4d5690059e682961cf0c [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:44.764495 15755 raft_consensus.cc:3037] T fa7601a7547a41ae85e5fadd135a00bd P aebc96dd8d22409caba2a0968488a3b8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:44.765786 15469 raft_consensus.cc:2454] T fa7601a7547a41ae85e5fadd135a00bd P fcf7f715e0ba4d5690059e682961cf0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 795c4878987947afa5d8ab2c58773e59 in term 1.
I20260807 05:09:44.765786 15755 raft_consensus.cc:2454] T fa7601a7547a41ae85e5fadd135a00bd P aebc96dd8d22409caba2a0968488a3b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 795c4878987947afa5d8ab2c58773e59 in term 1.
I20260807 05:09:44.766106 15542 leader_election.cc:302] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 795c4878987947afa5d8ab2c58773e59, aebc96dd8d22409caba2a0968488a3b8; no voters:
I20260807 05:09:44.766290 15846 raft_consensus.cc:2781] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:44.766510 15846 raft_consensus.cc:686] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59 [term 1 LEADER]: Becoming Leader. State: Replica: 795c4878987947afa5d8ab2c58773e59, State: Running, Role: LEADER
I20260807 05:09:44.766855 15846 consensus_queue.cc:237] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7f715e0ba4d5690059e682961cf0c" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 35079 } } peers { permanent_uuid: "aebc96dd8d22409caba2a0968488a3b8" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38737 } } peers { permanent_uuid: "795c4878987947afa5d8ab2c58773e59" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37209 } }
I20260807 05:09:44.770717 14506 catalog_manager.cc:5495] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 795c4878987947afa5d8ab2c58773e59 (127.8.16.66). New cstate: current_term: 1 leader_uuid: "795c4878987947afa5d8ab2c58773e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7f715e0ba4d5690059e682961cf0c" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 35079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aebc96dd8d22409caba2a0968488a3b8" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "795c4878987947afa5d8ab2c58773e59" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37209 } health_report { overall_health: HEALTHY } } }
I20260807 05:09:44.783421 15837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.780015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35218 (local address 127.8.16.66:37209)
0807 05:09:44.780136 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:44.780140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:44.780153 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:09:44.780284 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:44.780286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:44.780340 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:44.780427 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:44.780662 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.781163 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.782027 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.782246 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.782333 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.782660 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:44.783056 (+ 396us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:44.783160 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:44.783228 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 05:09:44.783293 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 05:09:44.788957 15755 raft_consensus.cc:1261] T fa7601a7547a41ae85e5fadd135a00bd P aebc96dd8d22409caba2a0968488a3b8 [term 1 FOLLOWER]: Refusing update from remote peer 795c4878987947afa5d8ab2c58773e59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:44.788957 15469 raft_consensus.cc:1261] T fa7601a7547a41ae85e5fadd135a00bd P fcf7f715e0ba4d5690059e682961cf0c [term 1 FOLLOWER]: Refusing update from remote peer 795c4878987947afa5d8ab2c58773e59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:44.789667 15846 consensus_queue.cc:1035] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf7f715e0ba4d5690059e682961cf0c" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 35079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:44.789855 15847 consensus_queue.cc:1035] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aebc96dd8d22409caba2a0968488a3b8" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:44.801105 15852 mvcc.cc:205] Tried to move back new op lower bound from 7315781160088522752 to 7315781160007548928. Current Snapshot: MvccSnapshot[applied={T|T < 7315781160088522752}]
I20260807 05:09:44.802650 15851 mvcc.cc:205] Tried to move back new op lower bound from 7315781160088522752 to 7315781160007548928. Current Snapshot: MvccSnapshot[applied={T|T < 7315781160088522752 or (T in {7315781160088522752})}]
I20260807 05:09:44.809839 14507 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:44.809975 14507 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:44.814265 14507 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81f0ce27c75145b3ace095da2ddb85d4: Sending DeleteTablet for 3 replicas of tablet fa7601a7547a41ae85e5fadd135a00bd
I20260807 05:09:44.815296 15448 tablet_service.cc:1514] Processing DeleteTablet for tablet fa7601a7547a41ae85e5fadd135a00bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:44 UTC) from {username='oryx', principal='oryx/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:46812
I20260807 05:09:44.815492 15735 tablet_service.cc:1514] Processing DeleteTablet for tablet fa7601a7547a41ae85e5fadd135a00bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:44 UTC) from {username='oryx', principal='oryx/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:56280
I20260807 05:09:44.815828 15862 tablet_replica.cc:330] T fa7601a7547a41ae85e5fadd135a00bd P fcf7f715e0ba4d5690059e682961cf0c: stopping tablet replica
I20260807 05:09:44.816094 15590 tablet_service.cc:1514] Processing DeleteTablet for tablet fa7601a7547a41ae85e5fadd135a00bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:44 UTC) from {username='oryx', principal='oryx/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:35204
I20260807 05:09:44.816541 15864 tablet_replica.cc:330] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59: stopping tablet replica
I20260807 05:09:44.816671 15863 tablet_replica.cc:330] T fa7601a7547a41ae85e5fadd135a00bd P aebc96dd8d22409caba2a0968488a3b8: stopping tablet replica
I20260807 05:09:44.816883 15864 raft_consensus.cc:2229] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:09:44.816885 15863 raft_consensus.cc:2229] T fa7601a7547a41ae85e5fadd135a00bd P aebc96dd8d22409caba2a0968488a3b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:44.817196 15863 raft_consensus.cc:2258] T fa7601a7547a41ae85e5fadd135a00bd P aebc96dd8d22409caba2a0968488a3b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:44.817233 15864 raft_consensus.cc:2258] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:44.818688 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 14473
I20260807 05:09:44.819995 15864 ts_tablet_manager.cc:1905] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:44.820927 15863 ts_tablet_manager.cc:1905] T fa7601a7547a41ae85e5fadd135a00bd P aebc96dd8d22409caba2a0968488a3b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:44.822737 15864 ts_tablet_manager.cc:1918] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:09:44.822825 15864 log.cc:1196] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/wal/wals/fa7601a7547a41ae85e5fadd135a00bd
I20260807 05:09:44.823118 15864 ts_tablet_manager.cc:1939] T fa7601a7547a41ae85e5fadd135a00bd P 795c4878987947afa5d8ab2c58773e59: Deleting consensus metadata
I20260807 05:09:44.823660 15863 ts_tablet_manager.cc:1918] T fa7601a7547a41ae85e5fadd135a00bd P aebc96dd8d22409caba2a0968488a3b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:09:44.823745 15863 log.cc:1196] T fa7601a7547a41ae85e5fadd135a00bd P aebc96dd8d22409caba2a0968488a3b8: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/fa7601a7547a41ae85e5fadd135a00bd
I20260807 05:09:44.824026 15863 ts_tablet_manager.cc:1939] T fa7601a7547a41ae85e5fadd135a00bd P aebc96dd8d22409caba2a0968488a3b8: Deleting consensus metadata
I20260807 05:09:44.825541 15862 raft_consensus.cc:2229] T fa7601a7547a41ae85e5fadd135a00bd P fcf7f715e0ba4d5690059e682961cf0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:44.825915 15862 raft_consensus.cc:2258] T fa7601a7547a41ae85e5fadd135a00bd P fcf7f715e0ba4d5690059e682961cf0c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 05:09:44.827059 15685 connection.cc:535] server connection from 127.0.0.1:56280 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 05:09:44.827042 15542 connection.cc:535] server connection from 127.0.0.1:35204 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 05:09:44.827618 15862 ts_tablet_manager.cc:1905] T fa7601a7547a41ae85e5fadd135a00bd P fcf7f715e0ba4d5690059e682961cf0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:44.830291 15862 ts_tablet_manager.cc:1918] T fa7601a7547a41ae85e5fadd135a00bd P fcf7f715e0ba4d5690059e682961cf0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:09:44.830392 15862 log.cc:1196] T fa7601a7547a41ae85e5fadd135a00bd P fcf7f715e0ba4d5690059e682961cf0c: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-0/wal/wals/fa7601a7547a41ae85e5fadd135a00bd
I20260807 05:09:44.830727 15862 ts_tablet_manager.cc:1939] T fa7601a7547a41ae85e5fadd135a00bd P fcf7f715e0ba4d5690059e682961cf0c: Deleting consensus metadata
W20260807 05:09:44.831677 15403 connection.cc:441] server connection from 127.0.0.1:46812 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46812 (request call id 1) could send its response
I20260807 05:09:44.837803 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.817602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46606 (local address 127.8.16.125:38779)
0807 05:09:44.817716 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:44.817720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:44.826428 (+ 8708us) server_negotiation.cc:407] Connection header received
0807 05:09:44.826550 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:44.826553 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:44.826605 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:44.826672 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:44.828224 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.828838 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.829755 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.830077 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.830180 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.832077 (+ 1897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:44.832080 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:44.832083 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:44.832108 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:44.833816 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.834347 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.834350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.834352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.834438 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.834705 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.834708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.834710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.834923 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:44.835040 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:44.836473 (+ 1433us) server_negotiation.cc:299] Negotiation successful
0807 05:09:44.837629 (+ 1156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 05:09:44.840610 15815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.830567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47918 (local address 127.8.16.124:44531)
0807 05:09:44.830684 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:44.830688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:44.832222 (+ 1534us) server_negotiation.cc:407] Connection header received
0807 05:09:44.832377 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:44.832380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:44.832429 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:44.832478 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:44.833374 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.834148 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.835287 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.835643 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.835713 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.837223 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:44.837227 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:44.837230 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:44.837266 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:44.838822 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.839276 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.839279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.839281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.839365 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.839669 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.839672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.839674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.839871 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:44.839968 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:44.840311 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 05:09:44.840439 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 05:09:44.840723 15818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.830264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46610 (local address 127.8.16.125:38779)
0807 05:09:44.830372 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:44.830377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:44.830706 (+ 329us) server_negotiation.cc:407] Connection header received
0807 05:09:44.830938 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:44.830941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:44.831009 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:44.831066 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:44.831956 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.832655 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.833719 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.834065 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.834142 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.835733 (+ 1591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:44.835737 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:44.835740 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:44.835762 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:44.837532 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.839478 (+ 1946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.839481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.839483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.839557 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.839846 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.839849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.839852 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.840048 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:44.840156 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:44.840480 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 05:09:44.840595 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 05:09:44.843847 15524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:44.818142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47910 (local address 127.8.16.124:44531)
0807 05:09:44.818246 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:44.818250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:44.835652 (+ 17402us) server_negotiation.cc:407] Connection header received
0807 05:09:44.835694 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:44.835696 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:44.835756 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:44.835821 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:44.836817 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.837594 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.838754 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:44.839114 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:44.839231 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:44.840691 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:44.840696 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:44.840698 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:44.840723 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:44.842493 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.842851 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.842854 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.842856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.842938 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:44.843137 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:44.843140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:44.843141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:44.843332 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:44.843456 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:44.843588 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 05:09:44.843689 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 05:09:45.802079 15872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:45.801600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.8.16.66:58393)
0807 05:09:45.801848 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:45.801963 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":80,"threads_started":1}
I20260807 05:09:45.802316 15873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:45.801826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.8.16.67:45513)
0807 05:09:45.802129 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:45.802223 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":66,"threads_started":1}
W20260807 05:09:45.802686 15802 heartbeater.cc:644] Failed to heartbeat to 127.8.16.126:46471 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
W20260807 05:09:45.802894 15660 heartbeater.cc:644] Failed to heartbeat to 127.8.16.126:46471 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
I20260807 05:09:45.803468 15874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:45.802940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.8.16.65:59155)
0807 05:09:45.803292 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:45.803377 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":65,"threads_started":1}
W20260807 05:09:45.803722 15517 heartbeater.cc:644] Failed to heartbeat to 127.8.16.126:46471 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
I20260807 05:09:46.314321 15875 raft_consensus.cc:482] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 81f0ce27c75145b3ace095da2ddb85d4)
I20260807 05:09:46.314544 15875 raft_consensus.cc:504] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: false } }
I20260807 05:09:46.315502 15876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 81f0ce27c75145b3ace095da2ddb85d4)
I20260807 05:09:46.315697 15876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: false } }
I20260807 05:09:46.316061 15875 leader_election.cc:288] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 81f0ce27c75145b3ace095da2ddb85d4 (127.8.16.126:46471), 8b93668c0b5f46a8ad3b2c6c983d461d (127.8.16.124:44531)
I20260807 05:09:46.316447 15878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.315831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.0.0.1:48670)
0807 05:09:46.316239 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.316320 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":93,"thread_start_us":54,"threads_started":1}
W20260807 05:09:46.316625 15126 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
I20260807 05:09:46.316790 15876 leader_election.cc:288] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 81f0ce27c75145b3ace095da2ddb85d4 (127.8.16.126:46471), 421e96d56e9f4523a75349dccff2cb5d (127.8.16.125:38779)
I20260807 05:09:46.318313 15878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.318045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.0.0.1:48676)
0807 05:09:46.318185 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.318246 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 05:09:46.318413 15883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.317518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.0.0.1:48674)
0807 05:09:46.318213 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.318279 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":63,"threads_started":1}
W20260807 05:09:46.318428 15126 leader_election.cc:334] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 81f0ce27c75145b3ace095da2ddb85d4 (127.8.16.126:46471): Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
W20260807 05:09:46.318583 15319 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
I20260807 05:09:46.319799 15883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.319535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.0.0.1:48684)
0807 05:09:46.319673 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.319736 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260807 05:09:46.319900 15319 leader_election.cc:334] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 81f0ce27c75145b3ace095da2ddb85d4 (127.8.16.126:46471): Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
I20260807 05:09:46.320986 15877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.315831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:44531 (local address 127.0.0.1:38858)
0807 05:09:46.316227 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.316242 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:46.316416 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:46.317025 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:46.317027 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:46.317044 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:46.317193 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:46.317200 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.318449 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.318453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:46.319629 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:46.319640 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.320694 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.320697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:46.320768 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:46.320787 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 05:09:46.320835 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 05:09:46.320893 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":72,"threads_started":1}
I20260807 05:09:46.321040 15879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.316016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38858 (local address 127.8.16.124:44531)
0807 05:09:46.316704 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:46.316709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:46.316723 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:09:46.316775 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:46.316778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:46.316827 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:46.316912 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:46.317395 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.318317 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.319962 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.320594 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.320700 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:46.320760 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:46.320859 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 05:09:46.320926 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":603,"thread_start_us":57,"threads_started":1}
I20260807 05:09:46.321499 15336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "421e96d56e9f4523a75349dccff2cb5d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" is_pre_election: true
I20260807 05:09:46.321681 15336 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 421e96d56e9f4523a75349dccff2cb5d in term 1.
I20260807 05:09:46.322034 15124 leader_election.cc:302] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 421e96d56e9f4523a75349dccff2cb5d, 8b93668c0b5f46a8ad3b2c6c983d461d; no voters: 81f0ce27c75145b3ace095da2ddb85d4
I20260807 05:09:46.322278 15875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 05:09:46.322340 15875 raft_consensus.cc:482] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 81f0ce27c75145b3ace095da2ddb85d4)
I20260807 05:09:46.322374 15881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.317169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:38779 (local address 127.0.0.1:36504)
0807 05:09:46.317884 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.317898 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:46.318875 (+ 977us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:46.319215 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:46.319218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:46.319248 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:46.319335 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:46.319341 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.320172 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.320175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:46.321283 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:46.321293 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.321987 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.321991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:46.322075 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:46.322098 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:09:46.322174 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 05:09:46.322257 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"mutex_wait_us":19,"thread_start_us":77,"threads_started":1}
I20260807 05:09:46.322393 15875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:09:46.322441 15884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.317610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36504 (local address 127.8.16.125:38779)
0807 05:09:46.318098 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:46.318107 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:46.318806 (+ 699us) server_negotiation.cc:407] Connection header received
0807 05:09:46.318981 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:46.318984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:46.319025 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:46.319076 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:46.319464 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.320090 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.321383 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.321890 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.321989 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:46.322042 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:46.322234 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 05:09:46.322308 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":97,"threads_started":1}
I20260807 05:09:46.322789 15150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "421e96d56e9f4523a75349dccff2cb5d" is_pre_election: true
I20260807 05:09:46.323526 15875 raft_consensus.cc:504] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: false } }
I20260807 05:09:46.323756 15150 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b93668c0b5f46a8ad3b2c6c983d461d in current term 2: Already voted for candidate 421e96d56e9f4523a75349dccff2cb5d in this term.
I20260807 05:09:46.323920 15875 leader_election.cc:288] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [CANDIDATE]: Term 2 election: Requested vote from peers 81f0ce27c75145b3ace095da2ddb85d4 (127.8.16.126:46471), 8b93668c0b5f46a8ad3b2c6c983d461d (127.8.16.124:44531)
I20260807 05:09:46.324276 15877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.324055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.0.0.1:48692)
0807 05:09:46.324161 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.324210 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 05:09:46.324368 15318 leader_election.cc:302] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b93668c0b5f46a8ad3b2c6c983d461d; no voters: 421e96d56e9f4523a75349dccff2cb5d, 81f0ce27c75145b3ace095da2ddb85d4
I20260807 05:09:46.324473 15336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "421e96d56e9f4523a75349dccff2cb5d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d"
I20260807 05:09:46.324566 15336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:09:46.324981 15877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.324763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.0.0.1:48704)
0807 05:09:46.324878 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.324927 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 05:09:46.325086 15126 leader_election.cc:334] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 81f0ce27c75145b3ace095da2ddb85d4 (127.8.16.126:46471): Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
I20260807 05:09:46.325680 15336 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 421e96d56e9f4523a75349dccff2cb5d in term 2.
I20260807 05:09:46.325780 15876 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:09:46.325977 15124 leader_election.cc:302] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 421e96d56e9f4523a75349dccff2cb5d, 8b93668c0b5f46a8ad3b2c6c983d461d; no voters: 81f0ce27c75145b3ace095da2ddb85d4
I20260807 05:09:46.326159 15875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 2 FOLLOWER]: Leader election won for term 2
I20260807 05:09:46.326367 15875 raft_consensus.cc:686] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [term 2 LEADER]: Becoming Leader. State: Replica: 421e96d56e9f4523a75349dccff2cb5d, State: Running, Role: LEADER
I20260807 05:09:46.326599 15875 consensus_queue.cc:237] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: false } }
I20260807 05:09:46.327535 15886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 421e96d56e9f4523a75349dccff2cb5d. Latest consensus state: current_term: 2 leader_uuid: "421e96d56e9f4523a75349dccff2cb5d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: false } } }
I20260807 05:09:46.327656 15886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:46.327996 15888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:46.330473 15888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:46.330861 15888 catalog_manager.cc:1259] Loaded cluster ID: 5f5f89a5f3fa42ecb10b2f77a3d886cc
I20260807 05:09:46.330907 15888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:46.331444 15888 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:46.331813 15888 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d: Loaded TSK: 0
I20260807 05:09:46.332471 15888 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:09:46.353513 15139 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:09:46.356477 15139 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:09:46.359440 15877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.359168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.0.0.1:48720)
0807 05:09:46.359291 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.359356 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 05:09:46.359478 15336 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [term 2 FOLLOWER]: Refusing update from remote peer 421e96d56e9f4523a75349dccff2cb5d: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 05:09:46.360008 15886 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 05:09:46.360396 15877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.360127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.0.0.1:48734)
0807 05:09:46.360277 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.360327 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260807 05:09:46.360528 15126 consensus_peers.cc:487] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d -> Peer 81f0ce27c75145b3ace095da2ddb85d4 (127.8.16.126:46471): Couldn't send request to peer 81f0ce27c75145b3ace095da2ddb85d4. Status: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 05:09:46.361613 15876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 421e96d56e9f4523a75349dccff2cb5d. Latest consensus state: current_term: 2 leader_uuid: "421e96d56e9f4523a75349dccff2cb5d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: false } } }
I20260807 05:09:46.361749 15876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:09:46.362969 15875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "421e96d56e9f4523a75349dccff2cb5d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: false } } }
I20260807 05:09:46.363090 15875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:46.363225 15876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "421e96d56e9f4523a75349dccff2cb5d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81f0ce27c75145b3ace095da2ddb85d4" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 46471 } } peers { permanent_uuid: "421e96d56e9f4523a75349dccff2cb5d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 38779 } attrs { promote: false } } peers { permanent_uuid: "8b93668c0b5f46a8ad3b2c6c983d461d" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 44531 } attrs { promote: false } } }
I20260807 05:09:46.363345 15876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b93668c0b5f46a8ad3b2c6c983d461d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:09:46.370309 15890 mvcc.cc:205] Tried to move back new op lower bound from 7315781166523830272 to 7315781166395846656. Current Snapshot: MvccSnapshot[applied={T|T < 7315781166523830272}]
I20260807 05:09:46.373816 15878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.368593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:38737 (local address 127.0.0.1:53404)
0807 05:09:46.369010 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.369026 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:46.369116 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:46.369819 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:46.369822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:46.369841 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:46.369934 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:46.369940 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.370905 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.370908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:46.371953 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:46.371962 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.372830 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.372834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:46.372932 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:46.373532 (+ 600us) client_negotiation.cc:769] Sending connection context
0807 05:09:46.373606 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 05:09:46.373677 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":553}
I20260807 05:09:46.374188 15894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.368759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53404 (local address 127.8.16.67:38737)
0807 05:09:46.369432 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:46.369436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:46.369495 (+ 59us) server_negotiation.cc:407] Connection header received
0807 05:09:46.369551 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:46.369554 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:46.369601 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:46.369655 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:46.370058 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.370790 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.372085 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.372711 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.372988 (+ 277us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:46.373051 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:46.373976 (+ 925us) server_negotiation.cc:299] Negotiation successful
0807 05:09:46.374054 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":90,"threads_started":1}
I20260807 05:09:46.374632 15735 tablet_service.cc:1467] Processing CreateTablet for tablet 893f76dc812347e6b3670d2bbe42a08e (DEFAULT_TABLE table=test-table [id=bbbe734b273b405982aa78de99bde530]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:46.374920 15735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 893f76dc812347e6b3670d2bbe42a08e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:46.375061 15877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.368819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:37209 (local address 127.0.0.1:49422)
0807 05:09:46.369392 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.369405 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:46.369502 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:46.371398 (+ 1896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:46.371401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:46.371418 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:46.371492 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:46.371497 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.372643 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.372647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:46.373839 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:46.373851 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.374704 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.374707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:46.374792 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:46.374816 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 05:09:46.374863 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 05:09:46.374935 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":2}
I20260807 05:09:46.375624 15895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.369062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49422 (local address 127.8.16.66:37209)
0807 05:09:46.369948 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:46.369952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:46.369965 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:09:46.370014 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:46.370017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:46.370061 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:46.371713 (+ 1652us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:46.371819 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.372547 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.373985 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.374577 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.375329 (+ 752us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:46.375390 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:46.375424 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 05:09:46.375490 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":809,"thread_start_us":80,"threads_started":1}
I20260807 05:09:46.376065 15590 tablet_service.cc:1467] Processing CreateTablet for tablet 893f76dc812347e6b3670d2bbe42a08e (DEFAULT_TABLE table=test-table [id=bbbe734b273b405982aa78de99bde530]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:46.376250 15896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.369052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:35079 (local address 127.0.0.1:41684)
0807 05:09:46.369676 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.369685 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:46.369751 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:46.370509 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:46.370512 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:46.370528 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:46.370631 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:46.370636 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.373152 (+ 2516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.373155 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:46.374487 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:46.374496 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.375801 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.375806 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:46.375905 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:46.375928 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:09:46.375982 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 05:09:46.376045 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":103,"threads_started":1}
I20260807 05:09:46.376339 15590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 893f76dc812347e6b3670d2bbe42a08e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:46.376864 15897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.369395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41684 (local address 127.8.16.65:35079)
0807 05:09:46.370283 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:46.370287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:46.370300 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:09:46.370350 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:46.370353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:46.370398 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:46.370718 (+ 320us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:46.370811 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.373018 (+ 2207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.374629 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.375537 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.376573 (+ 1036us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:46.376645 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:46.376678 (+ 33us) server_negotiation.cc:299] Negotiation successful
0807 05:09:46.376744 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":812,"thread_start_us":103,"threads_started":1}
I20260807 05:09:46.377409 15448 tablet_service.cc:1467] Processing CreateTablet for tablet 893f76dc812347e6b3670d2bbe42a08e (DEFAULT_TABLE table=test-table [id=bbbe734b273b405982aa78de99bde530]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:46.377704 15448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 893f76dc812347e6b3670d2bbe42a08e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:46.378852 15902 tablet_bootstrap.cc:492] T 893f76dc812347e6b3670d2bbe42a08e P 795c4878987947afa5d8ab2c58773e59: Bootstrap starting.
I20260807 05:09:46.379565 15902 tablet_bootstrap.cc:654] T 893f76dc812347e6b3670d2bbe42a08e P 795c4878987947afa5d8ab2c58773e59: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:46.379562 15903 tablet_bootstrap.cc:492] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c: Bootstrap starting.
I20260807 05:09:46.379976 15901 tablet_bootstrap.cc:492] T 893f76dc812347e6b3670d2bbe42a08e P aebc96dd8d22409caba2a0968488a3b8: Bootstrap starting.
I20260807 05:09:46.380385 15903 tablet_bootstrap.cc:654] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:46.380712 15902 tablet_bootstrap.cc:492] T 893f76dc812347e6b3670d2bbe42a08e P 795c4878987947afa5d8ab2c58773e59: No bootstrap required, opened a new log
I20260807 05:09:46.380798 15902 ts_tablet_manager.cc:1397] T 893f76dc812347e6b3670d2bbe42a08e P 795c4878987947afa5d8ab2c58773e59: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:46.381219 15902 raft_consensus.cc:348] T 893f76dc812347e6b3670d2bbe42a08e P 795c4878987947afa5d8ab2c58773e59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aebc96dd8d22409caba2a0968488a3b8" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38737 } } peers { permanent_uuid: "795c4878987947afa5d8ab2c58773e59" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37209 } } peers { permanent_uuid: "fcf7f715e0ba4d5690059e682961cf0c" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 35079 } }
I20260807 05:09:46.381291 15901 tablet_bootstrap.cc:654] T 893f76dc812347e6b3670d2bbe42a08e P aebc96dd8d22409caba2a0968488a3b8: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:46.381317 15902 raft_consensus.cc:374] T 893f76dc812347e6b3670d2bbe42a08e P 795c4878987947afa5d8ab2c58773e59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:46.381347 15902 raft_consensus.cc:729] T 893f76dc812347e6b3670d2bbe42a08e P 795c4878987947afa5d8ab2c58773e59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 795c4878987947afa5d8ab2c58773e59, State: Initialized, Role: FOLLOWER
I20260807 05:09:46.381481 15902 consensus_queue.cc:260] T 893f76dc812347e6b3670d2bbe42a08e P 795c4878987947afa5d8ab2c58773e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aebc96dd8d22409caba2a0968488a3b8" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38737 } } peers { permanent_uuid: "795c4878987947afa5d8ab2c58773e59" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37209 } } peers { permanent_uuid: "fcf7f715e0ba4d5690059e682961cf0c" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 35079 } }
I20260807 05:09:46.381588 15903 tablet_bootstrap.cc:492] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c: No bootstrap required, opened a new log
I20260807 05:09:46.381671 15903 ts_tablet_manager.cc:1397] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:46.381821 15902 ts_tablet_manager.cc:1428] T 893f76dc812347e6b3670d2bbe42a08e P 795c4878987947afa5d8ab2c58773e59: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:09:46.382138 15903 raft_consensus.cc:348] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aebc96dd8d22409caba2a0968488a3b8" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38737 } } peers { permanent_uuid: "795c4878987947afa5d8ab2c58773e59" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37209 } } peers { permanent_uuid: "fcf7f715e0ba4d5690059e682961cf0c" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 35079 } }
I20260807 05:09:46.382251 15903 raft_consensus.cc:374] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:46.382292 15903 raft_consensus.cc:729] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf7f715e0ba4d5690059e682961cf0c, State: Initialized, Role: FOLLOWER
I20260807 05:09:46.382427 15903 consensus_queue.cc:260] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aebc96dd8d22409caba2a0968488a3b8" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38737 } } peers { permanent_uuid: "795c4878987947afa5d8ab2c58773e59" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37209 } } peers { permanent_uuid: "fcf7f715e0ba4d5690059e682961cf0c" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 35079 } }
I20260807 05:09:46.382705 15908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.382326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.8.16.66:46805)
0807 05:09:46.382560 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.382617 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":94,"threads_started":1}
I20260807 05:09:46.382720 15903 ts_tablet_manager.cc:1428] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:09:46.384889 15908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.384665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.8.16.66:33617)
0807 05:09:46.384774 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.384825 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 05:09:46.385180 15519 heartbeater.cc:503] Master 127.8.16.125:38779 was elected leader, sending a full tablet report...
I20260807 05:09:46.385375 15662 heartbeater.cc:503] Master 127.8.16.125:38779 was elected leader, sending a full tablet report...
I20260807 05:09:46.385396 15910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.383063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.8.16.65:39487)
0807 05:09:46.385188 (+ 2125us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.385258 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2049,"thread_start_us":108,"threads_started":1}
I20260807 05:09:46.386190 15911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.385813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.8.16.65:44493)
0807 05:09:46.386032 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.386104 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":100,"threads_started":1}
I20260807 05:09:46.387557 15901 tablet_bootstrap.cc:492] T 893f76dc812347e6b3670d2bbe42a08e P aebc96dd8d22409caba2a0968488a3b8: No bootstrap required, opened a new log
I20260807 05:09:46.387660 15901 ts_tablet_manager.cc:1397] T 893f76dc812347e6b3670d2bbe42a08e P aebc96dd8d22409caba2a0968488a3b8: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260807 05:09:46.388012 15901 raft_consensus.cc:348] T 893f76dc812347e6b3670d2bbe42a08e P aebc96dd8d22409caba2a0968488a3b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aebc96dd8d22409caba2a0968488a3b8" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38737 } } peers { permanent_uuid: "795c4878987947afa5d8ab2c58773e59" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37209 } } peers { permanent_uuid: "fcf7f715e0ba4d5690059e682961cf0c" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 35079 } }
I20260807 05:09:46.388105 15901 raft_consensus.cc:374] T 893f76dc812347e6b3670d2bbe42a08e P aebc96dd8d22409caba2a0968488a3b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:46.388134 15901 raft_consensus.cc:729] T 893f76dc812347e6b3670d2bbe42a08e P aebc96dd8d22409caba2a0968488a3b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aebc96dd8d22409caba2a0968488a3b8, State: Initialized, Role: FOLLOWER
I20260807 05:09:46.388263 15901 consensus_queue.cc:260] T 893f76dc812347e6b3670d2bbe42a08e P aebc96dd8d22409caba2a0968488a3b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aebc96dd8d22409caba2a0968488a3b8" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38737 } } peers { permanent_uuid: "795c4878987947afa5d8ab2c58773e59" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37209 } } peers { permanent_uuid: "fcf7f715e0ba4d5690059e682961cf0c" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 35079 } }
I20260807 05:09:46.388587 15901 ts_tablet_manager.cc:1428] T 893f76dc812347e6b3670d2bbe42a08e P aebc96dd8d22409caba2a0968488a3b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:09:46.389266 15913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.388869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.8.16.67:60753)
0807 05:09:46.389087 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.389141 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":99,"threads_started":1}
I20260807 05:09:46.390192 15804 heartbeater.cc:503] Master 127.8.16.125:38779 was elected leader, sending a full tablet report...
I20260807 05:09:46.390436 15913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.390217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.8.16.67:47433)
0807 05:09:46.390324 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.390376 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 05:09:46.584287 15805 tablet.cc:2369] T 893f76dc812347e6b3670d2bbe42a08e P aebc96dd8d22409caba2a0968488a3b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 05:09:46.593429 15663 tablet.cc:2369] T 893f76dc812347e6b3670d2bbe42a08e P 795c4878987947afa5d8ab2c58773e59: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:09:46.654287 15909 raft_consensus.cc:482] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:09:46.654450 15909 raft_consensus.cc:504] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aebc96dd8d22409caba2a0968488a3b8" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38737 } } peers { permanent_uuid: "795c4878987947afa5d8ab2c58773e59" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37209 } } peers { permanent_uuid: "fcf7f715e0ba4d5690059e682961cf0c" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 35079 } }
I20260807 05:09:46.655531 15909 leader_election.cc:288] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aebc96dd8d22409caba2a0968488a3b8 (127.8.16.67:38737), 795c4878987947afa5d8ab2c58773e59 (127.8.16.66:37209)
I20260807 05:09:46.659399 15895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.655951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:38551 (local address 127.8.16.66:37209)
0807 05:09:46.656072 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:46.656077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:46.656140 (+ 63us) server_negotiation.cc:407] Connection header received
0807 05:09:46.656367 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:46.656371 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:46.656423 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:46.656492 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:46.656749 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.657245 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.658443 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.658836 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.658929 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:46.658988 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:46.659192 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 05:09:46.659267 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 05:09:46.659403 15910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.655757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:38737 (local address 127.8.16.65:49161)
0807 05:09:46.655953 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.655967 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:46.656074 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:46.656335 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:46.656338 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:46.656358 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:46.656493 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:46.656499 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.657286 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.657289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:46.658340 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:46.658350 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.658962 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.658966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:46.659092 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:46.659110 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 05:09:46.659180 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 05:09:46.659279 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":111}
I20260807 05:09:46.659406 15894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.655864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:49161 (local address 127.8.16.67:38737)
0807 05:09:46.655980 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:46.655985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:46.656027 (+ 42us) server_negotiation.cc:407] Connection header received
0807 05:09:46.656182 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:46.656185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:46.656235 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:46.656311 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:46.656652 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.657183 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.658460 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.658836 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.658923 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:46.658987 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:46.659207 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 05:09:46.659289 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 05:09:46.659406 15911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.655783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:37209 (local address 127.8.16.65:38551)
0807 05:09:46.655956 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.655970 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:46.656205 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:46.656517 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:46.656520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:46.656537 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:46.656642 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:46.656648 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.657348 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.657351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:46.658326 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:46.658336 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.658965 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.658969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:46.659081 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:46.659099 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 05:09:46.659158 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 05:09:46.659261 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":42}
I20260807 05:09:46.660068 15755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "893f76dc812347e6b3670d2bbe42a08e" candidate_uuid: "fcf7f715e0ba4d5690059e682961cf0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aebc96dd8d22409caba2a0968488a3b8" is_pre_election: true
I20260807 05:09:46.660223 15755 raft_consensus.cc:2454] T 893f76dc812347e6b3670d2bbe42a08e P aebc96dd8d22409caba2a0968488a3b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf7f715e0ba4d5690059e682961cf0c in term 0.
I20260807 05:09:46.660476 15611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "893f76dc812347e6b3670d2bbe42a08e" candidate_uuid: "fcf7f715e0ba4d5690059e682961cf0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "795c4878987947afa5d8ab2c58773e59" is_pre_election: true
I20260807 05:09:46.660627 15400 leader_election.cc:302] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aebc96dd8d22409caba2a0968488a3b8, fcf7f715e0ba4d5690059e682961cf0c; no voters:
I20260807 05:09:46.660729 15611 raft_consensus.cc:2454] T 893f76dc812347e6b3670d2bbe42a08e P 795c4878987947afa5d8ab2c58773e59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf7f715e0ba4d5690059e682961cf0c in term 0.
I20260807 05:09:46.660888 15909 raft_consensus.cc:2781] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:09:46.660955 15909 raft_consensus.cc:482] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:09:46.660991 15909 raft_consensus.cc:3037] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:46.662212 15909 raft_consensus.cc:504] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aebc96dd8d22409caba2a0968488a3b8" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38737 } } peers { permanent_uuid: "795c4878987947afa5d8ab2c58773e59" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37209 } } peers { permanent_uuid: "fcf7f715e0ba4d5690059e682961cf0c" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 35079 } }
I20260807 05:09:46.662503 15909 leader_election.cc:288] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c [CANDIDATE]: Term 1 election: Requested vote from peers aebc96dd8d22409caba2a0968488a3b8 (127.8.16.67:38737), 795c4878987947afa5d8ab2c58773e59 (127.8.16.66:37209)
I20260807 05:09:46.662801 15611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "893f76dc812347e6b3670d2bbe42a08e" candidate_uuid: "fcf7f715e0ba4d5690059e682961cf0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "795c4878987947afa5d8ab2c58773e59"
I20260807 05:09:46.662822 15755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "893f76dc812347e6b3670d2bbe42a08e" candidate_uuid: "fcf7f715e0ba4d5690059e682961cf0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aebc96dd8d22409caba2a0968488a3b8"
I20260807 05:09:46.662912 15611 raft_consensus.cc:3037] T 893f76dc812347e6b3670d2bbe42a08e P 795c4878987947afa5d8ab2c58773e59 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:46.662917 15755 raft_consensus.cc:3037] T 893f76dc812347e6b3670d2bbe42a08e P aebc96dd8d22409caba2a0968488a3b8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:46.663790 15755 raft_consensus.cc:2454] T 893f76dc812347e6b3670d2bbe42a08e P aebc96dd8d22409caba2a0968488a3b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf7f715e0ba4d5690059e682961cf0c in term 1.
I20260807 05:09:46.663826 15611 raft_consensus.cc:2454] T 893f76dc812347e6b3670d2bbe42a08e P 795c4878987947afa5d8ab2c58773e59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf7f715e0ba4d5690059e682961cf0c in term 1.
I20260807 05:09:46.664047 15400 leader_election.cc:302] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 795c4878987947afa5d8ab2c58773e59, aebc96dd8d22409caba2a0968488a3b8, fcf7f715e0ba4d5690059e682961cf0c; no voters:
I20260807 05:09:46.664278 15909 raft_consensus.cc:2781] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:46.664475 15909 raft_consensus.cc:686] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c [term 1 LEADER]: Becoming Leader. State: Replica: fcf7f715e0ba4d5690059e682961cf0c, State: Running, Role: LEADER
I20260807 05:09:46.664705 15909 consensus_queue.cc:237] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aebc96dd8d22409caba2a0968488a3b8" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38737 } } peers { permanent_uuid: "795c4878987947afa5d8ab2c58773e59" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37209 } } peers { permanent_uuid: "fcf7f715e0ba4d5690059e682961cf0c" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 35079 } }
I20260807 05:09:46.665081 15911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.664886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.8.16.65:53569)
0807 05:09:46.664972 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.665022 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 05:09:46.666936 15138 catalog_manager.cc:5495] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf7f715e0ba4d5690059e682961cf0c (127.8.16.65). New cstate: current_term: 1 leader_uuid: "fcf7f715e0ba4d5690059e682961cf0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aebc96dd8d22409caba2a0968488a3b8" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "795c4878987947afa5d8ab2c58773e59" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf7f715e0ba4d5690059e682961cf0c" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 35079 } health_report { overall_health: HEALTHY } } }
I20260807 05:09:46.741632 15897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.737902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41696 (local address 127.8.16.65:35079)
0807 05:09:46.738039 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:46.738044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:46.738060 (+ 16us) server_negotiation.cc:407] Connection header received
0807 05:09:46.738151 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:46.738154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:46.738222 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:46.738330 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:46.738518 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.739311 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.740129 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:46.740465 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:46.740553 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:46.740742 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:46.741268 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:46.741372 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:46.741403 (+ 31us) server_negotiation.cc:299] Negotiation successful
0807 05:09:46.741484 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 05:09:46.744423 15755 raft_consensus.cc:1261] T 893f76dc812347e6b3670d2bbe42a08e P aebc96dd8d22409caba2a0968488a3b8 [term 1 FOLLOWER]: Refusing update from remote peer fcf7f715e0ba4d5690059e682961cf0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:46.744625 15611 raft_consensus.cc:1261] T 893f76dc812347e6b3670d2bbe42a08e P 795c4878987947afa5d8ab2c58773e59 [term 1 FOLLOWER]: Refusing update from remote peer fcf7f715e0ba4d5690059e682961cf0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:46.744973 15914 consensus_queue.cc:1035] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aebc96dd8d22409caba2a0968488a3b8" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:46.745182 15909 consensus_queue.cc:1035] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "795c4878987947afa5d8ab2c58773e59" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:46.746622 15913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.746370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.8.16.67:42777)
0807 05:09:46.746475 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.746545 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 05:09:46.748318 15908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.747866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.8.16.66:58825)
0807 05:09:46.748183 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.748248 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248}
I20260807 05:09:46.750628 15911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.750395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.8.16.65:60795)
0807 05:09:46.750491 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.750547 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 05:09:46.751363 15913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.751050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.8.16.67:50215)
0807 05:09:46.751244 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.751298 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260807 05:09:46.751514 15911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.751264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.8.16.65:58643)
0807 05:09:46.751398 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.751448 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 05:09:46.753508 15908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.753277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:46471 (local address 127.8.16.66:38433)
0807 05:09:46.753365 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.753416 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 05:09:46.760077 15137 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:46.760257 15137 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:46.763877 15137 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 421e96d56e9f4523a75349dccff2cb5d: Sending DeleteTablet for 3 replicas of tablet 893f76dc812347e6b3670d2bbe42a08e
I20260807 05:09:46.764730 15735 tablet_service.cc:1514] Processing DeleteTablet for tablet 893f76dc812347e6b3670d2bbe42a08e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:46 UTC) from {username='oryx', principal='oryx/127.8.16.125@KRBTEST.COM'} at 127.0.0.1:53404
I20260807 05:09:46.764899 15590 tablet_service.cc:1514] Processing DeleteTablet for tablet 893f76dc812347e6b3670d2bbe42a08e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:46 UTC) from {username='oryx', principal='oryx/127.8.16.125@KRBTEST.COM'} at 127.0.0.1:49422
I20260807 05:09:46.765126 15927 tablet_replica.cc:330] T 893f76dc812347e6b3670d2bbe42a08e P aebc96dd8d22409caba2a0968488a3b8: stopping tablet replica
I20260807 05:09:46.765219 15448 tablet_service.cc:1514] Processing DeleteTablet for tablet 893f76dc812347e6b3670d2bbe42a08e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:46 UTC) from {username='oryx', principal='oryx/127.8.16.125@KRBTEST.COM'} at 127.0.0.1:41684
I20260807 05:09:46.765266 15927 raft_consensus.cc:2229] T 893f76dc812347e6b3670d2bbe42a08e P aebc96dd8d22409caba2a0968488a3b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:46.765401 15927 raft_consensus.cc:2258] T 893f76dc812347e6b3670d2bbe42a08e P aebc96dd8d22409caba2a0968488a3b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:46.765642 15929 tablet_replica.cc:330] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c: stopping tablet replica
I20260807 05:09:46.765806 15928 tablet_replica.cc:330] T 893f76dc812347e6b3670d2bbe42a08e P 795c4878987947afa5d8ab2c58773e59: stopping tablet replica
I20260807 05:09:46.765825 15929 raft_consensus.cc:2229] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:09:46.765923 15928 raft_consensus.cc:2229] T 893f76dc812347e6b3670d2bbe42a08e P 795c4878987947afa5d8ab2c58773e59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:46.766053 15928 raft_consensus.cc:2258] T 893f76dc812347e6b3670d2bbe42a08e P 795c4878987947afa5d8ab2c58773e59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:46.766281 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 15292
I20260807 05:09:46.766381 15929 raft_consensus.cc:2258] T 893f76dc812347e6b3670d2bbe42a08e P fcf7f715e0ba4d5690059e682961cf0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:46.767046 15927 ts_tablet_manager.cc:1905] T 893f76dc812347e6b3670d2bbe42a08e P aebc96dd8d22409caba2a0968488a3b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:46.767463 15928 ts_tablet_manager.cc:1905] T 893f76dc812347e6b3670d2bbe42a08e P 795c4878987947afa5d8ab2c58773e59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:46.769550 15928 ts_tablet_manager.cc:1918] T 893f76dc812347e6b3670d2bbe42a08e P 795c4878987947afa5d8ab2c58773e59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:09:46.769625 15928 log.cc:1196] T 893f76dc812347e6b3670d2bbe42a08e P 795c4878987947afa5d8ab2c58773e59: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-1/wal/wals/893f76dc812347e6b3670d2bbe42a08e
I20260807 05:09:46.769901 15928 ts_tablet_manager.cc:1939] T 893f76dc812347e6b3670d2bbe42a08e P 795c4878987947afa5d8ab2c58773e59: Deleting consensus metadata
I20260807 05:09:46.769901 15927 ts_tablet_manager.cc:1918] T 893f76dc812347e6b3670d2bbe42a08e P aebc96dd8d22409caba2a0968488a3b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:09:46.769964 15927 log.cc:1196] T 893f76dc812347e6b3670d2bbe42a08e P aebc96dd8d22409caba2a0968488a3b8: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/893f76dc812347e6b3670d2bbe42a08e
I20260807 05:09:46.770242 15927 ts_tablet_manager.cc:1939] T 893f76dc812347e6b3670d2bbe42a08e P aebc96dd8d22409caba2a0968488a3b8: Deleting consensus metadata
I20260807 05:09:46.770690 15125 catalog_manager.cc:4841] TS 795c4878987947afa5d8ab2c58773e59 (127.8.16.66:37209): tablet 893f76dc812347e6b3670d2bbe42a08e (table test-table [id=bbbe734b273b405982aa78de99bde530]) successfully deleted
I20260807 05:09:46.770974 15123 catalog_manager.cc:4841] TS aebc96dd8d22409caba2a0968488a3b8 (127.8.16.67:38737): tablet 893f76dc812347e6b3670d2bbe42a08e (table test-table [id=bbbe734b273b405982aa78de99bde530]) successfully deleted
I20260807 05:09:46.778430 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 15525
I20260807 05:09:46.779261 15896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:46.778486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:35079 (local address 127.0.0.1:41706)
0807 05:09:46.779102 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:46.779156 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.65:35079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":530}
W20260807 05:09:46.779728 15123 catalog_manager.cc:4584] TS fcf7f715e0ba4d5690059e682961cf0c (127.8.16.65:35079): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 893f76dc812347e6b3670d2bbe42a08e: Network error: Client connection negotiation failed: client connection to 127.8.16.65:35079: connect: Connection refused (error 111)
I20260807 05:09:46.784914 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 15667
I20260807 05:09:46.790485 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 14990
I20260807 05:09:46.796386 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 15104
2026-08-07T05:09:46Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5394 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:09:46 dist-test-slave-49h3 krb5kdc[15932](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:09:46 dist-test-slave-49h3 krb5kdc[15932](info): set up 2 sockets
Aug 07 05:09:46 dist-test-slave-49h3 krb5kdc[15932](info): commencing operation
krb5kdc: starting...
W20260807 05:09:47.781653 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.962s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:09:47 dist-test-slave-49h3 krb5kdc[15932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:09:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:47Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.126.keytab.
Entry for principal oryx/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.126.keytab.
I20260807 05:09:47.900156 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:38273
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.126.keytab
--principal=oryx/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:44837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:38273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:48.009995 15945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:48.010255 15945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:48.010311 15945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:48.014195 15945 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:48.014249 15945 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 05:09:48.014273 15945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:48.014297 15945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:48.014324 15945 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:48.014353 15945 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:48.019469 15945 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:44837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:38273
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.126.keytab
--principal=oryx/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:38273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:48.020474 15945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:48.021471 15945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:48.029774 15950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:48.029781 15951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:48.029780 15953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:48.029843 15945 server_base.cc:1034] running on GCE node
I20260807 05:09:48.030594 15945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:48.031584 15945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:48.032733 15945 hybrid_clock.cc:648] HybridClock initialized: now 1786079388032719 us; error 25 us; skew 500 ppm
Aug 07 05:09:48 dist-test-slave-49h3 krb5kdc[15932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079388, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:48.035347 15945 init.cc:377] Logged in from keytab as oryx/127.8.16.126@KRBTEST.COM (short username oryx)
I20260807 05:09:48.036170 15945 webserver.cc:460] Webserver started at http://127.8.16.126:37197/ using document root <none> and password file <none>
I20260807 05:09:48.036684 15945 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:48.036746 15945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:48.036931 15945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:48.038940 15945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "6e5f85b8e2fb4bb2aeba733336db713f"
format_stamp: "Formatted at 2026-08-07 05:09:48 on dist-test-slave-49h3"
server_key: "29d1bc46f3c75be43f3c363eee94ef3a"
server_key_iv: "a49ba326c0f585974ece85a8dea909ad"
server_key_version: "encryptionkey@0"
I20260807 05:09:48.039477 15945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "6e5f85b8e2fb4bb2aeba733336db713f"
format_stamp: "Formatted at 2026-08-07 05:09:48 on dist-test-slave-49h3"
server_key: "29d1bc46f3c75be43f3c363eee94ef3a"
server_key_iv: "a49ba326c0f585974ece85a8dea909ad"
server_key_version: "encryptionkey@0"
I20260807 05:09:48.043036 15945 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:09:48.045301 15962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:48.046619 15945 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:48.046722 15945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "6e5f85b8e2fb4bb2aeba733336db713f"
format_stamp: "Formatted at 2026-08-07 05:09:48 on dist-test-slave-49h3"
server_key: "29d1bc46f3c75be43f3c363eee94ef3a"
server_key_iv: "a49ba326c0f585974ece85a8dea909ad"
server_key_version: "encryptionkey@0"
I20260807 05:09:48.046809 15945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:48.063189 15945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:48.065873 15945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:48.066026 15945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:48.073376 15945 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:38273
I20260807 05:09:48.073437 16027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:38273 every 8 connection(s)
I20260807 05:09:48.074352 15945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:48.076450 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 15945
I20260807 05:09:48.076571 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:09:48.076850 8257 external_mini_cluster.cc:1428] Setting key 03fb966cd9ed71ce15161c14c4bec510
I20260807 05:09:48.077076 16028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:48.083974 16028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f: Bootstrap starting.
Aug 07 05:09:48 dist-test-slave-49h3 krb5kdc[15932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM
I20260807 05:09:48.086508 16028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:48.087306 16028 log.cc:824] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:48.089728 16028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f: No bootstrap required, opened a new log
I20260807 05:09:48.092741 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.078263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43188 (local address 127.8.16.126:38273)
0807 05:09:48.078667 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:48.078685 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:48.078728 (+ 43us) server_negotiation.cc:407] Connection header received
0807 05:09:48.079700 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:48.079722 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:48.080151 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:48.080506 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:48.081153 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.082636 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.083440 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.084113 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.084262 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.086910 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:48.086916 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:48.086932 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:48.086964 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:48.089651 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:48.090083 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:48.090089 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:48.090096 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:48.090241 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:48.090504 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:48.090507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:48.090509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:48.091012 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:48.091208 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:48.091491 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 05:09:48.091800 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":108,"threads_started":1}
I20260807 05:09:48.093576 16028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e5f85b8e2fb4bb2aeba733336db713f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38273 } }
I20260807 05:09:48.093837 16028 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:48.093894 16028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e5f85b8e2fb4bb2aeba733336db713f, State: Initialized, Role: FOLLOWER
I20260807 05:09:48.094321 16028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e5f85b8e2fb4bb2aeba733336db713f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38273 } }
I20260807 05:09:48.094449 16028 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:09:48.094512 16028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:09:48.094604 16028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:48.095469 16028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e5f85b8e2fb4bb2aeba733336db713f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38273 } }
I20260807 05:09:48.095775 16028 leader_election.cc:302] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e5f85b8e2fb4bb2aeba733336db713f; no voters:
I20260807 05:09:48.096076 16028 leader_election.cc:288] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:09:48.096215 16034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:48.096406 16034 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f [term 1 LEADER]: Becoming Leader. State: Replica: 6e5f85b8e2fb4bb2aeba733336db713f, State: Running, Role: LEADER
I20260807 05:09:48.096683 16034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e5f85b8e2fb4bb2aeba733336db713f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38273 } }
I20260807 05:09:48.097244 16028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:48.098459 16035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e5f85b8e2fb4bb2aeba733336db713f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e5f85b8e2fb4bb2aeba733336db713f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38273 } } }
I20260807 05:09:48.098580 16035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:48.098827 16037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e5f85b8e2fb4bb2aeba733336db713f. Latest consensus state: current_term: 1 leader_uuid: "6e5f85b8e2fb4bb2aeba733336db713f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e5f85b8e2fb4bb2aeba733336db713f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38273 } } }
I20260807 05:09:48.098917 16037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:48.099412 16044 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:48.101981 16044 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:48.106755 16044 catalog_manager.cc:1347] Generated new cluster ID: 488cbc38b0ca4a01b567a1fe58d88472
I20260807 05:09:48.106823 16044 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:48.140436 16044 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:09:48.141403 16044 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:48.149106 16044 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f: Generated new TSK 0
I20260807 05:09:48.149808 16044 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.65.keytab.
Entry for principal oryx/127.8.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.65.keytab.
I20260807 05:09:48.190482 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:38273
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.65.keytab
--principal=oryx/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:44837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:48.322045 16056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:48.322289 16056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:48.322330 16056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:48.327059 16056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:48.327121 16056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:48.327159 16056 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 05:09:48.327196 16056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:09:48.332340 16056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:44837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.65.keytab
--principal=oryx/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.16.126:38273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:48.333302 16056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:48.334306 16056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:48.343580 16064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:48.343590 16061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:48.343603 16062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:48.343703 16056 server_base.cc:1034] running on GCE node
I20260807 05:09:48.344229 16056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:48.344853 16056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:48.346005 16056 hybrid_clock.cc:648] HybridClock initialized: now 1786079388345986 us; error 30 us; skew 500 ppm
Aug 07 05:09:48 dist-test-slave-49h3 krb5kdc[15932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079388, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:48.348589 16056 init.cc:377] Logged in from keytab as oryx/127.8.16.65@KRBTEST.COM (short username oryx)
I20260807 05:09:48.349423 16056 webserver.cc:460] Webserver started at http://127.8.16.65:34619/ using document root <none> and password file <none>
I20260807 05:09:48.350010 16056 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:48.350077 16056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:48.350271 16056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:48.352288 16056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "52f3e3469f5d4c1cbe62d8084af37273"
format_stamp: "Formatted at 2026-08-07 05:09:48 on dist-test-slave-49h3"
server_key: "0d05fa6d1d9307a4e480a8e765271e55"
server_key_iv: "e522d72ac8a4c70754f18126839311bf"
server_key_version: "encryptionkey@0"
I20260807 05:09:48.352819 16056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "52f3e3469f5d4c1cbe62d8084af37273"
format_stamp: "Formatted at 2026-08-07 05:09:48 on dist-test-slave-49h3"
server_key: "0d05fa6d1d9307a4e480a8e765271e55"
server_key_iv: "e522d72ac8a4c70754f18126839311bf"
server_key_version: "encryptionkey@0"
I20260807 05:09:48.356336 16056 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:09:48.358652 16071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:48.359740 16056 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:48.359843 16056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "52f3e3469f5d4c1cbe62d8084af37273"
format_stamp: "Formatted at 2026-08-07 05:09:48 on dist-test-slave-49h3"
server_key: "0d05fa6d1d9307a4e480a8e765271e55"
server_key_iv: "e522d72ac8a4c70754f18126839311bf"
server_key_version: "encryptionkey@0"
I20260807 05:09:48.359929 16056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:48.384341 16056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:48.387176 16056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:48.387362 16056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:48.388901 16056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:48.388955 16056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:48.389010 16056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:48.389039 16056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 05:09:48 dist-test-slave-49h3 krb5kdc[15932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079388, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM
I20260807 05:09:48.399413 16056 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:39621
I20260807 05:09:48.400266 16056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:09:48.401023 16189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:39621 every 8 connection(s)
I20260807 05:09:48.402441 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.389771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:35701 (local address 127.8.16.126:38273)
0807 05:09:48.389887 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:48.389891 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:48.390916 (+ 1025us) server_negotiation.cc:407] Connection header received
0807 05:09:48.391871 (+ 955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:48.391875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:48.391935 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:48.391992 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:48.393472 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.394309 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.395534 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.395791 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.395884 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.398898 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:48.398903 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:48.398906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:48.398940 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:48.400446 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:48.401030 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:48.401033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:48.401035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:48.401126 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:48.401424 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:48.401426 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:48.401428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:48.401645 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:48.401722 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:48.402211 (+ 489us) server_negotiation.cc:299] Negotiation successful
0807 05:09:48.402319 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 05:09:48.403198 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.390288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:38273 (local address 127.8.16.65:35701)
0807 05:09:48.390772 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:48.390815 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:48.391629 (+ 814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:48.392396 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:48.392403 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:48.392747 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:48.393276 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:48.393292 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.394451 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.394454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:48.395390 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:48.395400 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.395952 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.395957 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:48.396123 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.396705 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:48.396727 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:48.398666 (+ 1939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:48.400557 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:48.400562 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:48.400577 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:48.400910 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:48.401206 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:48.401210 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:48.401212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:48.401344 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:48.401738 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:48.401743 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:48.401966 (+ 223us) client_negotiation.cc:769] Sending connection context
0807 05:09:48.402149 (+ 183us) client_negotiation.cc:240] Negotiation successful
0807 05:09:48.402394 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":139,"threads_started":1}
I20260807 05:09:48.404435 16190 heartbeater.cc:347] Connected to a master server at 127.8.16.126:38273
I20260807 05:09:48.404568 16190 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:48.405030 16190 heartbeater.cc:511] Master 127.8.16.126:38273 requested a full tablet report, sending...
I20260807 05:09:48.406513 15978 ts_manager.cc:194] Registered new tserver with Master: 52f3e3469f5d4c1cbe62d8084af37273 (127.8.16.65:39621)
I20260807 05:09:48.407593 15978 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:35701
I20260807 05:09:48.408226 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 16056
I20260807 05:09:48.408362 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:09:48.408700 8257 external_mini_cluster.cc:1428] Setting key 272fd04737b92d8eceaa82cd4f0d347f
I20260807 05:09:48.413100 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.406430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:35095 (local address 127.8.16.126:38273)
0807 05:09:48.406551 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:48.406555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:48.406769 (+ 214us) server_negotiation.cc:407] Connection header received
0807 05:09:48.406868 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:48.406871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:48.406913 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:48.406976 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:48.407375 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.408007 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.408959 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.409247 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.409334 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.410499 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:48.410503 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:48.410505 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:48.410526 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:48.411664 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:48.412112 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:48.412115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:48.412116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:48.412197 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:48.412445 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:48.412447 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:48.412448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:48.412597 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:48.412665 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:48.412899 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 05:09:48.412989 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:09:48.413314 16196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.406356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:38273 (local address 127.8.16.65:35095)
0807 05:09:48.406684 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:48.406698 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:48.406798 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:48.406998 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:48.407001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:48.407184 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:48.407276 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:48.407281 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.408135 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.408138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:48.408856 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:48.408865 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.409340 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.409344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:48.409433 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.409982 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:48.409996 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:48.410380 (+ 384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:48.411760 (+ 1380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:48.411764 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:48.411766 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:48.412002 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:48.412276 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:48.412280 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:48.412281 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:48.412371 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:48.412668 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:48.412671 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:48.412759 (+ 88us) client_negotiation.cc:769] Sending connection context
0807 05:09:48.412865 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 05:09:48.413004 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for oryx/127.8.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.66.keytab.
Entry for principal oryx/127.8.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.66.keytab.
I20260807 05:09:48.434635 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:38273
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.66.keytab
--principal=oryx/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:44837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:48.542167 16199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:48.542407 16199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:48.542448 16199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:48.546388 16199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:48.546443 16199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:48.546484 16199 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 05:09:48.546533 16199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:09:48.551647 16199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:44837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.66.keytab
--principal=oryx/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.16.126:38273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:48.552608 16199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:48.553610 16199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:48.562490 16207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:48.562600 16199 server_base.cc:1034] running on GCE node
W20260807 05:09:48.562496 16206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:48.562786 16209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:48.563184 16199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:48.563813 16199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:48.564973 16199 hybrid_clock.cc:648] HybridClock initialized: now 1786079388564957 us; error 27 us; skew 500 ppm
Aug 07 05:09:48 dist-test-slave-49h3 krb5kdc[15932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079388, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:48.567549 16199 init.cc:377] Logged in from keytab as oryx/127.8.16.66@KRBTEST.COM (short username oryx)
I20260807 05:09:48.568338 16199 webserver.cc:460] Webserver started at http://127.8.16.66:36103/ using document root <none> and password file <none>
I20260807 05:09:48.568847 16199 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:48.568912 16199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:48.569085 16199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:48.571090 16199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "196776ad987c4d29afccd36feb109f94"
format_stamp: "Formatted at 2026-08-07 05:09:48 on dist-test-slave-49h3"
server_key: "8a153c15686318771898c880a7aef83c"
server_key_iv: "9d00db6e98ebd3739f80ae22f5cbe3d3"
server_key_version: "encryptionkey@0"
I20260807 05:09:48.571606 16199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "196776ad987c4d29afccd36feb109f94"
format_stamp: "Formatted at 2026-08-07 05:09:48 on dist-test-slave-49h3"
server_key: "8a153c15686318771898c880a7aef83c"
server_key_iv: "9d00db6e98ebd3739f80ae22f5cbe3d3"
server_key_version: "encryptionkey@0"
I20260807 05:09:48.575079 16199 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260807 05:09:48.577224 16219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:48.578344 16199 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260807 05:09:48.578449 16199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "196776ad987c4d29afccd36feb109f94"
format_stamp: "Formatted at 2026-08-07 05:09:48 on dist-test-slave-49h3"
server_key: "8a153c15686318771898c880a7aef83c"
server_key_iv: "9d00db6e98ebd3739f80ae22f5cbe3d3"
server_key_version: "encryptionkey@0"
I20260807 05:09:48.578536 16199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:48.593851 16199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:48.596799 16199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:48.596968 16199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:48.598435 16199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:48.598487 16199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:48.598539 16199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:48.598568 16199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:48.608174 16199 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:33539
I20260807 05:09:48.608250 16342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:33539 every 8 connection(s)
Aug 07 05:09:48 dist-test-slave-49h3 krb5kdc[15932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079388, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM
I20260807 05:09:48.609100 16199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:09:48.610872 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 16199
I20260807 05:09:48.611001 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:09:48.611303 8257 external_mini_cluster.cc:1428] Setting key a03f163f4249325d32b2e2aa8d84d216
I20260807 05:09:48.612402 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.599340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:49331 (local address 127.8.16.126:38273)
0807 05:09:48.599475 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:48.599479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:48.600490 (+ 1011us) server_negotiation.cc:407] Connection header received
0807 05:09:48.601578 (+ 1088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:48.601582 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:48.601640 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:48.601697 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:48.603283 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.604123 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.605771 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.606132 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.606254 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.608865 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:48.608869 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:48.608872 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:48.608901 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:48.610420 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:48.610889 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:48.610893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:48.610895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:48.611001 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:48.611334 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:48.611337 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:48.611339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:48.611554 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:48.611653 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:48.612129 (+ 476us) server_negotiation.cc:299] Negotiation successful
0807 05:09:48.612245 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 05:09:48.613147 16228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.599898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:38273 (local address 127.8.16.66:49331)
0807 05:09:48.600354 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:48.600394 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:48.601311 (+ 917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:48.602094 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:48.602102 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:48.602497 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:48.603127 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:48.603147 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.604259 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.604263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:48.605638 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:48.605648 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.606273 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.606277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:48.606464 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.607102 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:48.607122 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:48.608692 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:48.610513 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:48.610517 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:48.610530 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:48.610794 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:48.611107 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:48.611110 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:48.611111 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:48.611245 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:48.611646 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:48.611652 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:48.611929 (+ 277us) client_negotiation.cc:769] Sending connection context
0807 05:09:48.612076 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 05:09:48.612311 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":122,"threads_started":1}
I20260807 05:09:48.614490 16343 heartbeater.cc:347] Connected to a master server at 127.8.16.126:38273
I20260807 05:09:48.614629 16343 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:48.615168 16343 heartbeater.cc:511] Master 127.8.16.126:38273 requested a full tablet report, sending...
I20260807 05:09:48.616159 15979 ts_manager.cc:194] Registered new tserver with Master: 196776ad987c4d29afccd36feb109f94 (127.8.16.66:33539)
I20260807 05:09:48.616866 15979 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:49331
WARNING: no policy specified for oryx/127.8.16.67@KRBTEST.COM; defaulting to no policy
I20260807 05:09:48.623039 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.616512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:53969 (local address 127.8.16.126:38273)
0807 05:09:48.616616 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:48.616619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:48.616831 (+ 212us) server_negotiation.cc:407] Connection header received
0807 05:09:48.616920 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:48.616923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:48.616965 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:48.617031 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:48.617620 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.618145 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.618962 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.619167 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.619257 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.620248 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:48.620252 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:48.620254 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:48.620276 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:48.621546 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:48.621918 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:48.621921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:48.621923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:48.621999 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:48.622243 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:48.622247 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:48.622249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:48.622442 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:48.622525 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:48.622797 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 05:09:48.622896 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 05:09:48.623050 16350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.616460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:38273 (local address 127.8.16.66:53969)
0807 05:09:48.616751 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:48.616763 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:48.616851 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:48.617047 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:48.617049 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:48.617370 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:48.617502 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:48.617510 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.618267 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.618270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:48.618862 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:48.618871 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.619262 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.619265 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:48.619364 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.619797 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:48.619811 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:48.620119 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:48.621643 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:48.621646 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:48.621648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:48.621842 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:48.622077 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:48.622080 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:48.622082 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:48.622159 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:48.622531 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:48.622534 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:48.622618 (+ 84us) client_negotiation.cc:769] Sending connection context
0807 05:09:48.622740 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 05:09:48.622875 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":95,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.67.keytab.
Entry for principal oryx/127.8.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.67.keytab.
I20260807 05:09:48.638151 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:38273
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.67.keytab
--principal=oryx/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:44837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:48.746044 16352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:48.746332 16352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:48.746377 16352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:48.750383 16352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:48.750447 16352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:48.750483 16352 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 05:09:48.750521 16352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:09:48.755878 16352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:44837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.67.keytab
--principal=oryx/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.16.126:38273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:48.756922 16352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:48.757968 16352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:48.767611 16363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:48.767616 16358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:48.767671 16360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:48.768031 16352 server_base.cc:1034] running on GCE node
I20260807 05:09:48.768461 16352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:48.769115 16352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:48.770299 16352 hybrid_clock.cc:648] HybridClock initialized: now 1786079388770280 us; error 30 us; skew 500 ppm
Aug 07 05:09:48 dist-test-slave-49h3 krb5kdc[15932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079388, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:48.772956 16352 init.cc:377] Logged in from keytab as oryx/127.8.16.67@KRBTEST.COM (short username oryx)
I20260807 05:09:48.773869 16352 webserver.cc:460] Webserver started at http://127.8.16.67:35701/ using document root <none> and password file <none>
I20260807 05:09:48.774381 16352 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:48.774438 16352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:48.774621 16352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:48.776598 16352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "2fb9a84dbaa84054b1607e6759c47250"
format_stamp: "Formatted at 2026-08-07 05:09:48 on dist-test-slave-49h3"
server_key: "074bbfcb53401d3f2ed95fd8b5cecdaf"
server_key_iv: "96ab4222115864eba159186746a0c4f7"
server_key_version: "encryptionkey@0"
I20260807 05:09:48.777107 16352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "2fb9a84dbaa84054b1607e6759c47250"
format_stamp: "Formatted at 2026-08-07 05:09:48 on dist-test-slave-49h3"
server_key: "074bbfcb53401d3f2ed95fd8b5cecdaf"
server_key_iv: "96ab4222115864eba159186746a0c4f7"
server_key_version: "encryptionkey@0"
I20260807 05:09:48.780797 16352 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:09:48.783147 16370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:48.784282 16352 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:48.784387 16352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "2fb9a84dbaa84054b1607e6759c47250"
format_stamp: "Formatted at 2026-08-07 05:09:48 on dist-test-slave-49h3"
server_key: "074bbfcb53401d3f2ed95fd8b5cecdaf"
server_key_iv: "96ab4222115864eba159186746a0c4f7"
server_key_version: "encryptionkey@0"
I20260807 05:09:48.784471 16352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:48.796811 16352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:48.799744 16352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:48.799896 16352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:48.801208 16352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:48.801254 16352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:48.801293 16352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:48.801319 16352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 05:09:48 dist-test-slave-49h3 krb5kdc[15932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079388, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM
I20260807 05:09:48.811395 16352 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:42769
I20260807 05:09:48.811532 16489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:42769 every 8 connection(s)
I20260807 05:09:48.812382 16352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:09:48.813999 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 16352
I20260807 05:09:48.814129 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:09:48.814472 8257 external_mini_cluster.cc:1428] Setting key 2d6195e1796a371504f375f29fe4e785
I20260807 05:09:48.815291 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.801966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:38939 (local address 127.8.16.126:38273)
0807 05:09:48.802077 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:48.802081 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:48.803091 (+ 1010us) server_negotiation.cc:407] Connection header received
0807 05:09:48.804083 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:48.804087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:48.804146 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:48.804204 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:48.805940 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.806767 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.808428 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.808749 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.808834 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.811697 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:48.811700 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:48.811703 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:48.811740 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:48.813165 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:48.813703 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:48.813706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:48.813707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:48.813815 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:48.814145 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:48.814149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:48.814150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:48.814392 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:48.814488 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:48.814990 (+ 502us) server_negotiation.cc:299] Negotiation successful
0807 05:09:48.815109 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 05:09:48.816121 16379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.802482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:38273 (local address 127.8.16.67:38939)
0807 05:09:48.802956 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:48.802996 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:48.803839 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:48.804625 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:48.804634 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:48.805112 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:48.805786 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:48.805807 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.806909 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.806913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:48.808297 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:48.808309 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.808909 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.808913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:48.809113 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.809818 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:48.809839 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:48.811522 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:48.813267 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:48.813272 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:48.813285 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:48.813611 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:48.813892 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:48.813894 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:48.813895 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:48.814014 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:48.814482 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:48.814487 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:48.814735 (+ 248us) client_negotiation.cc:769] Sending connection context
0807 05:09:48.814912 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 05:09:48.815202 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":142,"threads_started":1}
I20260807 05:09:48.817220 16490 heartbeater.cc:347] Connected to a master server at 127.8.16.126:38273
I20260807 05:09:48.817361 16490 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:48.817884 16490 heartbeater.cc:511] Master 127.8.16.126:38273 requested a full tablet report, sending...
I20260807 05:09:48.818884 15978 ts_manager.cc:194] Registered new tserver with Master: 2fb9a84dbaa84054b1607e6759c47250 (127.8.16.67:42769)
I20260807 05:09:48.819416 15978 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:38939
I20260807 05:09:48.825381 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.819482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:49175 (local address 127.8.16.126:38273)
0807 05:09:48.819582 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:48.819585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:48.819799 (+ 214us) server_negotiation.cc:407] Connection header received
0807 05:09:48.819870 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:48.819873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:48.819919 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:48.819989 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:48.820450 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.820942 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.821823 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.822020 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.822089 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.823119 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:48.823121 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:48.823124 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:48.823139 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:48.824210 (+ 1071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:48.824548 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:48.824550 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:48.824551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:48.824612 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:48.824802 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:48.824804 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:48.824805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:48.824947 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:48.825013 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:48.825189 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 05:09:48.825271 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 05:09:48.825389 16496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.819421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:38273 (local address 127.8.16.67:49175)
0807 05:09:48.819719 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:48.819733 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:48.819814 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:48.820027 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:48.820030 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:48.820249 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:48.820355 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:48.820362 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.821062 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.821065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:48.821738 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:48.821747 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.822133 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.822137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:48.822221 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.822696 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:48.822708 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:48.823015 (+ 307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:48.824294 (+ 1279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:48.824296 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:48.824298 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:48.824475 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:48.824668 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:48.824670 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:48.824671 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:48.824744 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:48.825007 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:48.825009 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:48.825077 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 05:09:48.825146 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 05:09:48.825250 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":104,"threads_started":1}
I20260807 05:09:48.828121 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 05:09:48 dist-test-slave-49h3 krb5kdc[15932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079388, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 05:09:48 dist-test-slave-49h3 krb5kdc[15932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:48 dist-test-slave-49h3 krb5kdc[15932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 05:09:48.850795 16032 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:48.845069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43192 (local address 127.8.16.126:38273)
0807 05:09:48.845182 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:48.845185 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:48.845198 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:09:48.845240 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:48.845243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:48.845287 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:48.845355 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:48.846088 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.846624 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.847393 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.847597 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.847662 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.850625 (+ 2963us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43192: BlockingRecv error: recv got EOF from 127.0.0.1:43192 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 07 05:09:48 dist-test-slave-49h3 krb5kdc[15932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079388, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM
I20260807 05:09:48.859301 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.852103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43200 (local address 127.8.16.126:38273)
0807 05:09:48.852234 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:48.852239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:48.852322 (+ 83us) server_negotiation.cc:407] Connection header received
0807 05:09:48.852436 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:48.852439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:48.852478 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:48.852541 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:48.853110 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.853637 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.854361 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.854564 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.854646 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.856620 (+ 1974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:48.856623 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:48.856625 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:48.856647 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:48.857838 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:48.858233 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:48.858237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:48.858238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:48.858311 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:48.858548 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:48.858552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:48.858553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:48.858719 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:48.858835 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:48.859049 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 05:09:48.859158 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 05:09:48.861810 15978 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:09:48.864867 15978 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:09:48.878283 16515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.872656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:39621 (local address 127.0.0.1:49732)
0807 05:09:48.873609 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:48.873660 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:48.873782 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:48.874493 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:48.874499 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:48.874522 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:48.874654 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:48.874661 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.875775 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.875779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:48.876945 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:48.876955 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.877751 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.877754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:48.877861 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.877899 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 05:09:48.878004 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 05:09:48.878137 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":87,"threads_started":1}
I20260807 05:09:48.878297 16513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.872806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49732 (local address 127.8.16.65:39621)
0807 05:09:48.873941 (+ 1135us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:48.873957 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:48.873980 (+ 23us) server_negotiation.cc:407] Connection header received
0807 05:09:48.874054 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:48.874058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:48.874254 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:48.874492 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:48.874774 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.875668 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.877063 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.877641 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.877738 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.877911 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:48.878042 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 05:09:48.878136 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1032,"thread_start_us":83,"threads_started":1}
I20260807 05:09:48.879142 16514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.872818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:42769 (local address 127.0.0.1:58194)
0807 05:09:48.874029 (+ 1211us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:48.874040 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:48.874109 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:48.874890 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:48.874893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:48.874910 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:48.875023 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:48.875030 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.876380 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.876383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:48.877651 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:48.877663 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.878773 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.878777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:48.878865 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.878888 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:09:48.878942 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 05:09:48.879002 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1157,"mutex_wait_us":38,"thread_start_us":100,"threads_started":1}
I20260807 05:09:48.879221 16517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.873400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58194 (local address 127.8.16.67:42769)
0807 05:09:48.874504 (+ 1104us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:48.874519 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:48.874539 (+ 20us) server_negotiation.cc:407] Connection header received
0807 05:09:48.874604 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:48.874608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:48.874775 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:48.874904 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:48.875125 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.876263 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.877789 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.878660 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.878763 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.878917 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:48.878984 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 05:09:48.879074 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1004,"thread_start_us":128,"threads_started":1}
I20260807 05:09:48.880707 16512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.873204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:33539 (local address 127.0.0.1:35250)
0807 05:09:48.874220 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:48.874229 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:48.874297 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:48.874934 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:48.874936 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:48.874955 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:48.875111 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:48.875118 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.876585 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.876588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:48.877729 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:48.877745 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.880323 (+ 2578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.880326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:48.880412 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.880432 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 05:09:48.880500 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 05:09:48.880569 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":977,"mutex_wait_us":16,"thread_start_us":100,"threads_started":1}
I20260807 05:09:48.880812 16518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.873512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35250 (local address 127.8.16.66:33539)
0807 05:09:48.873880 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:48.873893 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:48.874366 (+ 473us) server_negotiation.cc:407] Connection header received
0807 05:09:48.874459 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:48.874464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:48.874645 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:48.874789 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:48.875233 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.876472 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.877856 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.879116 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.880056 (+ 940us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.880213 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:48.880561 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 05:09:48.880666 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":84,"threads_started":1}
I20260807 05:09:48.881669 16421 tablet_service.cc:1467] Processing CreateTablet for tablet b7d501c3748a437d9e8edebd17048baa (DEFAULT_TABLE table=test-table [id=dcfb30ae0d724fd39e64c868fbfc18bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:48.881544 16123 tablet_service.cc:1467] Processing CreateTablet for tablet b7d501c3748a437d9e8edebd17048baa (DEFAULT_TABLE table=test-table [id=dcfb30ae0d724fd39e64c868fbfc18bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:48.882642 16421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7d501c3748a437d9e8edebd17048baa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:48.882642 16123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7d501c3748a437d9e8edebd17048baa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:48.883360 16273 tablet_service.cc:1467] Processing CreateTablet for tablet b7d501c3748a437d9e8edebd17048baa (DEFAULT_TABLE table=test-table [id=dcfb30ae0d724fd39e64c868fbfc18bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:48.884361 16273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7d501c3748a437d9e8edebd17048baa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:48.889894 16519 tablet_bootstrap.cc:492] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250: Bootstrap starting.
I20260807 05:09:48.891907 16519 tablet_bootstrap.cc:654] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:48.891980 16521 tablet_bootstrap.cc:492] T b7d501c3748a437d9e8edebd17048baa P 196776ad987c4d29afccd36feb109f94: Bootstrap starting.
I20260807 05:09:48.892536 16520 tablet_bootstrap.cc:492] T b7d501c3748a437d9e8edebd17048baa P 52f3e3469f5d4c1cbe62d8084af37273: Bootstrap starting.
I20260807 05:09:48.892782 16519 log.cc:824] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:48.894083 16521 tablet_bootstrap.cc:654] T b7d501c3748a437d9e8edebd17048baa P 196776ad987c4d29afccd36feb109f94: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:48.894508 16520 tablet_bootstrap.cc:654] T b7d501c3748a437d9e8edebd17048baa P 52f3e3469f5d4c1cbe62d8084af37273: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:48.894520 16519 tablet_bootstrap.cc:492] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250: No bootstrap required, opened a new log
I20260807 05:09:48.894706 16519 ts_tablet_manager.cc:1397] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:09:48.894884 16521 log.cc:824] T b7d501c3748a437d9e8edebd17048baa P 196776ad987c4d29afccd36feb109f94: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:48.895274 16520 log.cc:824] T b7d501c3748a437d9e8edebd17048baa P 52f3e3469f5d4c1cbe62d8084af37273: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:48.896631 16521 tablet_bootstrap.cc:492] T b7d501c3748a437d9e8edebd17048baa P 196776ad987c4d29afccd36feb109f94: No bootstrap required, opened a new log
I20260807 05:09:48.896819 16521 ts_tablet_manager.cc:1397] T b7d501c3748a437d9e8edebd17048baa P 196776ad987c4d29afccd36feb109f94: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 05:09:48.897217 16520 tablet_bootstrap.cc:492] T b7d501c3748a437d9e8edebd17048baa P 52f3e3469f5d4c1cbe62d8084af37273: No bootstrap required, opened a new log
I20260807 05:09:48.897380 16520 ts_tablet_manager.cc:1397] T b7d501c3748a437d9e8edebd17048baa P 52f3e3469f5d4c1cbe62d8084af37273: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:09:48.898417 16519 raft_consensus.cc:348] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } }
I20260807 05:09:48.898696 16519 raft_consensus.cc:374] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:48.898756 16519 raft_consensus.cc:729] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fb9a84dbaa84054b1607e6759c47250, State: Initialized, Role: FOLLOWER
I20260807 05:09:48.899186 16519 consensus_queue.cc:260] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } }
I20260807 05:09:48.899852 16490 heartbeater.cc:503] Master 127.8.16.126:38273 was elected leader, sending a full tablet report...
I20260807 05:09:48.900206 16519 ts_tablet_manager.cc:1428] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 05:09:48.900347 16521 raft_consensus.cc:348] T b7d501c3748a437d9e8edebd17048baa P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } }
I20260807 05:09:48.900611 16521 raft_consensus.cc:374] T b7d501c3748a437d9e8edebd17048baa P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:48.900660 16521 raft_consensus.cc:729] T b7d501c3748a437d9e8edebd17048baa P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196776ad987c4d29afccd36feb109f94, State: Initialized, Role: FOLLOWER
I20260807 05:09:48.900913 16520 raft_consensus.cc:348] T b7d501c3748a437d9e8edebd17048baa P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } }
I20260807 05:09:48.901125 16520 raft_consensus.cc:374] T b7d501c3748a437d9e8edebd17048baa P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:48.901160 16520 raft_consensus.cc:729] T b7d501c3748a437d9e8edebd17048baa P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52f3e3469f5d4c1cbe62d8084af37273, State: Initialized, Role: FOLLOWER
I20260807 05:09:48.901093 16521 consensus_queue.cc:260] T b7d501c3748a437d9e8edebd17048baa P 196776ad987c4d29afccd36feb109f94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } }
I20260807 05:09:48.901503 16520 consensus_queue.cc:260] T b7d501c3748a437d9e8edebd17048baa P 52f3e3469f5d4c1cbe62d8084af37273 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } }
I20260807 05:09:48.902048 16521 ts_tablet_manager.cc:1428] T b7d501c3748a437d9e8edebd17048baa P 196776ad987c4d29afccd36feb109f94: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 05:09:48.902099 16190 heartbeater.cc:503] Master 127.8.16.126:38273 was elected leader, sending a full tablet report...
I20260807 05:09:48.902359 16520 ts_tablet_manager.cc:1428] T b7d501c3748a437d9e8edebd17048baa P 52f3e3469f5d4c1cbe62d8084af37273: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 05:09:48.902354 16343 heartbeater.cc:503] Master 127.8.16.126:38273 was elected leader, sending a full tablet report...
I20260807 05:09:48.919064 16525 raft_consensus.cc:482] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:09:48.919314 16525 raft_consensus.cc:504] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } }
I20260807 05:09:48.920372 16525 leader_election.cc:288] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52f3e3469f5d4c1cbe62d8084af37273 (127.8.16.65:39621), 196776ad987c4d29afccd36feb109f94 (127.8.16.66:33539)
I20260807 05:09:48.924010 16513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.920631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:36787 (local address 127.8.16.65:39621)
0807 05:09:48.920761 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:48.920765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:48.920876 (+ 111us) server_negotiation.cc:407] Connection header received
0807 05:09:48.920971 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:48.920973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:48.921015 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:48.921084 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:48.921339 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.921883 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.922949 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.923537 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.923646 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.923711 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:48.923852 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 05:09:48.923912 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 05:09:48.924044 16379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.920578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:39621 (local address 127.8.16.67:36787)
0807 05:09:48.920770 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:48.920783 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:48.920872 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:48.921107 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:48.921110 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:48.921128 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:48.921243 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:48.921248 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.921982 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.921985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:48.922820 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:48.922830 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.923641 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.923644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:48.923739 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.923764 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 05:09:48.923838 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 05:09:48.923912 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":34}
I20260807 05:09:48.924753 16518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.920696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:50061 (local address 127.8.16.66:33539)
0807 05:09:48.920830 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:48.920834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:48.921065 (+ 231us) server_negotiation.cc:407] Connection header received
0807 05:09:48.921167 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:48.921170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:48.921211 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:48.921269 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:48.921527 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.922015 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.923754 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.924294 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.924395 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.924450 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:48.924605 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 05:09:48.924655 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 05:09:48.924700 16143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7d501c3748a437d9e8edebd17048baa" candidate_uuid: "2fb9a84dbaa84054b1607e6759c47250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52f3e3469f5d4c1cbe62d8084af37273" is_pre_election: true
I20260807 05:09:48.924800 16528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.920587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:33539 (local address 127.8.16.67:50061)
0807 05:09:48.920985 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:48.920998 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:48.921079 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:48.921284 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:48.921287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:48.921301 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:48.921418 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:48.921423 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.922119 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.922122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:48.923637 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:48.923647 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.924405 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.924408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:48.924501 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.924522 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 05:09:48.924608 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 05:09:48.924670 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":52,"thread_start_us":133,"threads_started":1}
I20260807 05:09:48.924995 16143 raft_consensus.cc:2454] T b7d501c3748a437d9e8edebd17048baa P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fb9a84dbaa84054b1607e6759c47250 in term 0.
I20260807 05:09:48.925405 16295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7d501c3748a437d9e8edebd17048baa" candidate_uuid: "2fb9a84dbaa84054b1607e6759c47250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196776ad987c4d29afccd36feb109f94" is_pre_election: true
I20260807 05:09:48.925472 16373 leader_election.cc:302] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fb9a84dbaa84054b1607e6759c47250, 52f3e3469f5d4c1cbe62d8084af37273; no voters:
I20260807 05:09:48.925699 16295 raft_consensus.cc:2454] T b7d501c3748a437d9e8edebd17048baa P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fb9a84dbaa84054b1607e6759c47250 in term 0.
I20260807 05:09:48.925714 16525 raft_consensus.cc:2781] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:09:48.925787 16525 raft_consensus.cc:482] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:09:48.925829 16525 raft_consensus.cc:3037] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:48.926888 16525 raft_consensus.cc:504] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } }
I20260807 05:09:48.927208 16525 leader_election.cc:288] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250 [CANDIDATE]: Term 1 election: Requested vote from peers 52f3e3469f5d4c1cbe62d8084af37273 (127.8.16.65:39621), 196776ad987c4d29afccd36feb109f94 (127.8.16.66:33539)
I20260807 05:09:48.927482 16295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7d501c3748a437d9e8edebd17048baa" candidate_uuid: "2fb9a84dbaa84054b1607e6759c47250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196776ad987c4d29afccd36feb109f94"
I20260807 05:09:48.927501 16143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7d501c3748a437d9e8edebd17048baa" candidate_uuid: "2fb9a84dbaa84054b1607e6759c47250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52f3e3469f5d4c1cbe62d8084af37273"
I20260807 05:09:48.927582 16295 raft_consensus.cc:3037] T b7d501c3748a437d9e8edebd17048baa P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:48.927604 16143 raft_consensus.cc:3037] T b7d501c3748a437d9e8edebd17048baa P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:48.928539 16143 raft_consensus.cc:2454] T b7d501c3748a437d9e8edebd17048baa P 52f3e3469f5d4c1cbe62d8084af37273 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fb9a84dbaa84054b1607e6759c47250 in term 1.
I20260807 05:09:48.928800 16373 leader_election.cc:302] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fb9a84dbaa84054b1607e6759c47250, 52f3e3469f5d4c1cbe62d8084af37273; no voters:
I20260807 05:09:48.928973 16525 raft_consensus.cc:2781] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:48.929003 16295 raft_consensus.cc:2454] T b7d501c3748a437d9e8edebd17048baa P 196776ad987c4d29afccd36feb109f94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fb9a84dbaa84054b1607e6759c47250 in term 1.
I20260807 05:09:48.929175 16525 raft_consensus.cc:686] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250 [term 1 LEADER]: Becoming Leader. State: Replica: 2fb9a84dbaa84054b1607e6759c47250, State: Running, Role: LEADER
I20260807 05:09:48.929519 16525 consensus_queue.cc:237] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } }
I20260807 05:09:48.932348 15977 catalog_manager.cc:5495] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2fb9a84dbaa84054b1607e6759c47250 (127.8.16.67). New cstate: current_term: 1 leader_uuid: "2fb9a84dbaa84054b1607e6759c47250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } health_report { overall_health: UNKNOWN } } }
I20260807 05:09:48.943007 16517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:48.939926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58196 (local address 127.8.16.67:42769)
0807 05:09:48.940039 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:48.940043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:48.940056 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:09:48.940119 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:48.940121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:48.940171 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:48.940255 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:48.940444 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.940968 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.941822 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:48.942032 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:48.942109 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:48.942316 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:48.942718 (+ 402us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:48.942807 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:48.942850 (+ 43us) server_negotiation.cc:299] Negotiation successful
0807 05:09:48.942900 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 05:09:48.948257 16295 raft_consensus.cc:1261] T b7d501c3748a437d9e8edebd17048baa P 196776ad987c4d29afccd36feb109f94 [term 1 FOLLOWER]: Refusing update from remote peer 2fb9a84dbaa84054b1607e6759c47250: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:48.948264 16143 raft_consensus.cc:1261] T b7d501c3748a437d9e8edebd17048baa P 52f3e3469f5d4c1cbe62d8084af37273 [term 1 FOLLOWER]: Refusing update from remote peer 2fb9a84dbaa84054b1607e6759c47250: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:48.948864 16529 consensus_queue.cc:1035] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:48.949038 16525 consensus_queue.cc:1035] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:48.957026 16534 mvcc.cc:205] Tried to move back new op lower bound from 7315781177125896192 to 7315781177057898496. Current Snapshot: MvccSnapshot[applied={T|T < 7315781177125896192}]
I20260807 05:09:48.957219 16533 mvcc.cc:205] Tried to move back new op lower bound from 7315781177125896192 to 7315781177057898496. Current Snapshot: MvccSnapshot[applied={T|T < 7315781177125896192}]
I20260807 05:09:48.964051 15978 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:48.964197 15978 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:48.966459 15978 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f: Sending DeleteTablet for 3 replicas of tablet b7d501c3748a437d9e8edebd17048baa
I20260807 05:09:48.967170 16123 tablet_service.cc:1514] Processing DeleteTablet for tablet b7d501c3748a437d9e8edebd17048baa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:48 UTC) from {username='oryx', principal='oryx/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:49732
I20260807 05:09:48.967176 16421 tablet_service.cc:1514] Processing DeleteTablet for tablet b7d501c3748a437d9e8edebd17048baa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:48 UTC) from {username='oryx', principal='oryx/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:58194
I20260807 05:09:48.967182 16273 tablet_service.cc:1514] Processing DeleteTablet for tablet b7d501c3748a437d9e8edebd17048baa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:48 UTC) from {username='oryx', principal='oryx/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:35250
I20260807 05:09:48.967670 16543 tablet_replica.cc:330] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250: stopping tablet replica
I20260807 05:09:48.967895 16545 tablet_replica.cc:330] T b7d501c3748a437d9e8edebd17048baa P 196776ad987c4d29afccd36feb109f94: stopping tablet replica
I20260807 05:09:48.968190 16545 raft_consensus.cc:2229] T b7d501c3748a437d9e8edebd17048baa P 196776ad987c4d29afccd36feb109f94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:48.968504 16545 raft_consensus.cc:2258] T b7d501c3748a437d9e8edebd17048baa P 196776ad987c4d29afccd36feb109f94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:48.968505 15978 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:09:48.968892 15978 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:09:48.968948 16543 raft_consensus.cc:2229] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:09:48.969324 16543 raft_consensus.cc:2258] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:48.969691 16542 tablet_replica.cc:330] T b7d501c3748a437d9e8edebd17048baa P 52f3e3469f5d4c1cbe62d8084af37273: stopping tablet replica
I20260807 05:09:48.969951 16542 raft_consensus.cc:2229] T b7d501c3748a437d9e8edebd17048baa P 52f3e3469f5d4c1cbe62d8084af37273 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:48.970233 16542 raft_consensus.cc:2258] T b7d501c3748a437d9e8edebd17048baa P 52f3e3469f5d4c1cbe62d8084af37273 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:48.971088 16543 ts_tablet_manager.cc:1905] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:48.971632 16545 ts_tablet_manager.cc:1905] T b7d501c3748a437d9e8edebd17048baa P 196776ad987c4d29afccd36feb109f94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:48.974121 16123 tablet_service.cc:1467] Processing CreateTablet for tablet 48a967eacd564edfa93c896b309f4f4a (DEFAULT_TABLE table=test-table [id=a27675e86de74ab4a3002f665c36edf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:48.974319 16273 tablet_service.cc:1467] Processing CreateTablet for tablet 48a967eacd564edfa93c896b309f4f4a (DEFAULT_TABLE table=test-table [id=a27675e86de74ab4a3002f665c36edf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:48.974408 16123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48a967eacd564edfa93c896b309f4f4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:48.974541 16273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48a967eacd564edfa93c896b309f4f4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:48.975363 16543 ts_tablet_manager.cc:1918] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:09:48.975522 16543 log.cc:1196] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/b7d501c3748a437d9e8edebd17048baa
I20260807 05:09:48.975924 16543 ts_tablet_manager.cc:1939] T b7d501c3748a437d9e8edebd17048baa P 2fb9a84dbaa84054b1607e6759c47250: Deleting consensus metadata
I20260807 05:09:48.976613 16521 tablet_bootstrap.cc:492] T 48a967eacd564edfa93c896b309f4f4a P 196776ad987c4d29afccd36feb109f94: Bootstrap starting.
I20260807 05:09:48.976680 16520 tablet_bootstrap.cc:492] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273: Bootstrap starting.
I20260807 05:09:48.977591 16521 tablet_bootstrap.cc:654] T 48a967eacd564edfa93c896b309f4f4a P 196776ad987c4d29afccd36feb109f94: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:48.977626 16520 tablet_bootstrap.cc:654] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:48.978327 16421 tablet_service.cc:1467] Processing CreateTablet for tablet 48a967eacd564edfa93c896b309f4f4a (DEFAULT_TABLE table=test-table [id=a27675e86de74ab4a3002f665c36edf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:48.978559 16421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48a967eacd564edfa93c896b309f4f4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:48.978855 16542 ts_tablet_manager.cc:1905] T b7d501c3748a437d9e8edebd17048baa P 52f3e3469f5d4c1cbe62d8084af37273: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:48.979040 16545 ts_tablet_manager.cc:1918] T b7d501c3748a437d9e8edebd17048baa P 196776ad987c4d29afccd36feb109f94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:09:48.979136 16545 log.cc:1196] T b7d501c3748a437d9e8edebd17048baa P 196776ad987c4d29afccd36feb109f94: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-1/wal/wals/b7d501c3748a437d9e8edebd17048baa
I20260807 05:09:48.979431 16545 ts_tablet_manager.cc:1939] T b7d501c3748a437d9e8edebd17048baa P 196776ad987c4d29afccd36feb109f94: Deleting consensus metadata
I20260807 05:09:48.980644 15963 catalog_manager.cc:4841] TS 196776ad987c4d29afccd36feb109f94 (127.8.16.66:33539): tablet b7d501c3748a437d9e8edebd17048baa (table test-table [id=dcfb30ae0d724fd39e64c868fbfc18bc]) successfully deleted
I20260807 05:09:48.981014 16519 tablet_bootstrap.cc:492] T 48a967eacd564edfa93c896b309f4f4a P 2fb9a84dbaa84054b1607e6759c47250: Bootstrap starting.
I20260807 05:09:48.981161 16542 ts_tablet_manager.cc:1918] T b7d501c3748a437d9e8edebd17048baa P 52f3e3469f5d4c1cbe62d8084af37273: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:09:48.981223 16542 log.cc:1196] T b7d501c3748a437d9e8edebd17048baa P 52f3e3469f5d4c1cbe62d8084af37273: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-0/wal/wals/b7d501c3748a437d9e8edebd17048baa
I20260807 05:09:48.981858 16542 ts_tablet_manager.cc:1939] T b7d501c3748a437d9e8edebd17048baa P 52f3e3469f5d4c1cbe62d8084af37273: Deleting consensus metadata
I20260807 05:09:48.982053 16519 tablet_bootstrap.cc:654] T 48a967eacd564edfa93c896b309f4f4a P 2fb9a84dbaa84054b1607e6759c47250: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:48.982782 15965 catalog_manager.cc:4841] TS 52f3e3469f5d4c1cbe62d8084af37273 (127.8.16.65:39621): tablet b7d501c3748a437d9e8edebd17048baa (table test-table [id=dcfb30ae0d724fd39e64c868fbfc18bc]) successfully deleted
I20260807 05:09:48.983605 16519 tablet_bootstrap.cc:492] T 48a967eacd564edfa93c896b309f4f4a P 2fb9a84dbaa84054b1607e6759c47250: No bootstrap required, opened a new log
I20260807 05:09:48.983660 15966 catalog_manager.cc:4841] TS 2fb9a84dbaa84054b1607e6759c47250 (127.8.16.67:42769): tablet b7d501c3748a437d9e8edebd17048baa (table test-table [id=dcfb30ae0d724fd39e64c868fbfc18bc]) successfully deleted
I20260807 05:09:48.983696 16519 ts_tablet_manager.cc:1397] T 48a967eacd564edfa93c896b309f4f4a P 2fb9a84dbaa84054b1607e6759c47250: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 05:09:48.984062 16519 raft_consensus.cc:348] T 48a967eacd564edfa93c896b309f4f4a P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:48.984158 16519 raft_consensus.cc:374] T 48a967eacd564edfa93c896b309f4f4a P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:48.984179 16519 raft_consensus.cc:729] T 48a967eacd564edfa93c896b309f4f4a P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fb9a84dbaa84054b1607e6759c47250, State: Initialized, Role: FOLLOWER
I20260807 05:09:48.984277 16519 consensus_queue.cc:260] T 48a967eacd564edfa93c896b309f4f4a P 2fb9a84dbaa84054b1607e6759c47250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:48.984328 16521 tablet_bootstrap.cc:492] T 48a967eacd564edfa93c896b309f4f4a P 196776ad987c4d29afccd36feb109f94: No bootstrap required, opened a new log
I20260807 05:09:48.984407 16521 ts_tablet_manager.cc:1397] T 48a967eacd564edfa93c896b309f4f4a P 196776ad987c4d29afccd36feb109f94: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20260807 05:09:48.984509 16519 ts_tablet_manager.cc:1428] T 48a967eacd564edfa93c896b309f4f4a P 2fb9a84dbaa84054b1607e6759c47250: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 05:09:48.984819 16521 raft_consensus.cc:348] T 48a967eacd564edfa93c896b309f4f4a P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:48.984925 16521 raft_consensus.cc:374] T 48a967eacd564edfa93c896b309f4f4a P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:48.984953 16521 raft_consensus.cc:729] T 48a967eacd564edfa93c896b309f4f4a P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196776ad987c4d29afccd36feb109f94, State: Initialized, Role: FOLLOWER
I20260807 05:09:48.985064 16521 consensus_queue.cc:260] T 48a967eacd564edfa93c896b309f4f4a P 196776ad987c4d29afccd36feb109f94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:48.985301 16521 ts_tablet_manager.cc:1428] T 48a967eacd564edfa93c896b309f4f4a P 196776ad987c4d29afccd36feb109f94: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 05:09:48.987411 16520 tablet_bootstrap.cc:492] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273: No bootstrap required, opened a new log
I20260807 05:09:48.987486 16520 ts_tablet_manager.cc:1397] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20260807 05:09:48.987851 16520 raft_consensus.cc:348] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:48.987938 16520 raft_consensus.cc:374] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:48.987959 16520 raft_consensus.cc:729] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52f3e3469f5d4c1cbe62d8084af37273, State: Initialized, Role: FOLLOWER
I20260807 05:09:48.988070 16520 consensus_queue.cc:260] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:48.988305 16520 ts_tablet_manager.cc:1428] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 05:09:49.032526 16527 raft_consensus.cc:482] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:09:49.032686 16527 raft_consensus.cc:504] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.033812 16527 leader_election.cc:288] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 196776ad987c4d29afccd36feb109f94 (127.8.16.66:33539), 2fb9a84dbaa84054b1607e6759c47250 (127.8.16.67:42769)
I20260807 05:09:49.038491 16517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:49.034492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:58811 (local address 127.8.16.67:42769)
0807 05:09:49.034633 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:49.034637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:49.034653 (+ 16us) server_negotiation.cc:407] Connection header received
0807 05:09:49.034733 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:49.034736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:49.034785 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:49.034862 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:49.035152 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.035921 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.037532 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.037939 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.038036 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:49.038101 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:49.038301 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 05:09:49.038367 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 05:09:49.038520 16550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:49.034067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:42769 (local address 127.8.16.65:58811)
0807 05:09:49.034522 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:49.034534 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:49.034638 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:49.034892 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:49.034895 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:49.034915 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:49.035043 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:49.035050 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.036024 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.036028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:49.037232 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:49.037244 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.038056 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.038060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:49.038184 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:49.038205 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 05:09:49.038272 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 05:09:49.038348 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":36,"spinlock_wait_cycles":24320,"thread_start_us":116,"threads_started":1}
I20260807 05:09:49.038738 16518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:49.034111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:51193 (local address 127.8.16.66:33539)
0807 05:09:49.034295 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:49.034300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:49.034377 (+ 77us) server_negotiation.cc:407] Connection header received
0807 05:09:49.034797 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:49.034801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:49.034848 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:49.034913 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:09:49.035232 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.035714 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.037367 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.038018 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.038121 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:49.038183 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:49.038530 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 05:09:49.038610 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260807 05:09:49.039096 16549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:49.034014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:33539 (local address 127.8.16.65:51193)
0807 05:09:49.034292 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:49.034306 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:49.034639 (+ 333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:49.035012 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:49.035015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:49.035027 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:09:49.035143 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:49.035149 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.035819 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.035823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:49.037234 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:49.037244 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.038263 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.038266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:49.038388 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:49.038412 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 05:09:49.038871 (+ 459us) client_negotiation.cc:240] Negotiation successful
0807 05:09:49.038950 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":50,"spinlock_wait_cycles":7040,"thread_start_us":83,"threads_started":1}
I20260807 05:09:49.039264 16442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48a967eacd564edfa93c896b309f4f4a" candidate_uuid: "52f3e3469f5d4c1cbe62d8084af37273" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fb9a84dbaa84054b1607e6759c47250" is_pre_election: true
I20260807 05:09:49.039530 16442 raft_consensus.cc:2454] T 48a967eacd564edfa93c896b309f4f4a P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52f3e3469f5d4c1cbe62d8084af37273 in term 0.
I20260807 05:09:49.039505 16295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48a967eacd564edfa93c896b309f4f4a" candidate_uuid: "52f3e3469f5d4c1cbe62d8084af37273" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196776ad987c4d29afccd36feb109f94" is_pre_election: true
I20260807 05:09:49.039726 16295 raft_consensus.cc:2454] T 48a967eacd564edfa93c896b309f4f4a P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52f3e3469f5d4c1cbe62d8084af37273 in term 0.
I20260807 05:09:49.040097 16072 leader_election.cc:302] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196776ad987c4d29afccd36feb109f94, 52f3e3469f5d4c1cbe62d8084af37273; no voters:
I20260807 05:09:49.040382 16527 raft_consensus.cc:2781] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:09:49.040446 16527 raft_consensus.cc:482] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:09:49.040481 16527 raft_consensus.cc:3037] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:49.041433 16527 raft_consensus.cc:504] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.041787 16527 leader_election.cc:288] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273 [CANDIDATE]: Term 1 election: Requested vote from peers 196776ad987c4d29afccd36feb109f94 (127.8.16.66:33539), 2fb9a84dbaa84054b1607e6759c47250 (127.8.16.67:42769)
I20260807 05:09:49.042138 16442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48a967eacd564edfa93c896b309f4f4a" candidate_uuid: "52f3e3469f5d4c1cbe62d8084af37273" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fb9a84dbaa84054b1607e6759c47250"
I20260807 05:09:49.042138 16295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48a967eacd564edfa93c896b309f4f4a" candidate_uuid: "52f3e3469f5d4c1cbe62d8084af37273" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196776ad987c4d29afccd36feb109f94"
I20260807 05:09:49.042256 16442 raft_consensus.cc:3037] T 48a967eacd564edfa93c896b309f4f4a P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:49.042263 16295 raft_consensus.cc:3037] T 48a967eacd564edfa93c896b309f4f4a P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:49.043315 16442 raft_consensus.cc:2454] T 48a967eacd564edfa93c896b309f4f4a P 2fb9a84dbaa84054b1607e6759c47250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52f3e3469f5d4c1cbe62d8084af37273 in term 1.
I20260807 05:09:49.043315 16295 raft_consensus.cc:2454] T 48a967eacd564edfa93c896b309f4f4a P 196776ad987c4d29afccd36feb109f94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52f3e3469f5d4c1cbe62d8084af37273 in term 1.
I20260807 05:09:49.043596 16072 leader_election.cc:302] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196776ad987c4d29afccd36feb109f94, 52f3e3469f5d4c1cbe62d8084af37273; no voters:
I20260807 05:09:49.043758 16527 raft_consensus.cc:2781] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:49.043991 16527 raft_consensus.cc:686] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273 [term 1 LEADER]: Becoming Leader. State: Replica: 52f3e3469f5d4c1cbe62d8084af37273, State: Running, Role: LEADER
I20260807 05:09:49.044262 16527 consensus_queue.cc:237] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.046355 15979 catalog_manager.cc:5495] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52f3e3469f5d4c1cbe62d8084af37273 (127.8.16.65). New cstate: current_term: 1 leader_uuid: "52f3e3469f5d4c1cbe62d8084af37273" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } health_report { overall_health: UNKNOWN } } }
W20260807 05:09:49.063320 16491 tablet.cc:2369] T 48a967eacd564edfa93c896b309f4f4a P 2fb9a84dbaa84054b1607e6759c47250: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 05:09:49 dist-test-slave-49h3 krb5kdc[15932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079388, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.126@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM
I20260807 05:09:49.100029 16558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:49.091147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:38273 (local address 127.0.0.1:43204)
0807 05:09:49.091465 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:49.091481 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:49.091601 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:49.091840 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:49.091843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:49.092372 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:49.092518 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:49.092528 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.093221 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.093225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:49.094148 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:49.094159 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.094573 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.094577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:49.094672 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:49.095228 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:49.095241 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:49.096753 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:49.098609 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:49.098617 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:49.098622 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:49.098853 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:49.099104 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:49.099106 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:49.099107 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:49.099181 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:49.099553 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:49.099559 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:49.099670 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 05:09:49.099766 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 05:09:49.099893 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":104,"threads_started":1}
I20260807 05:09:49.100158 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:49.091227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43204 (local address 127.8.16.126:38273)
0807 05:09:49.091363 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:49.091368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:49.091555 (+ 187us) server_negotiation.cc:407] Connection header received
0807 05:09:49.091691 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:49.091694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:49.091733 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:49.091798 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:49.092621 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.093120 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.094273 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.094477 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.094546 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:49.096874 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:49.096877 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:49.096879 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:49.096899 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:49.098519 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:49.098944 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:49.098948 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:49.098950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:49.099032 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:49.099253 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:49.099256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:49.099258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:49.099469 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:49.099555 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:49.099824 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 05:09:49.099968 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":58}
I20260807 05:09:49.103830 15979 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:43204:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 05:09:49.104620 15979 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:09:49.109421 16421 tablet_service.cc:1467] Processing CreateTablet for tablet 247b67f292464f82be23c8cc6f011ec9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=07c774bb2e964b3a85a8e539d2cc1b8d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 05:09:49.109421 16123 tablet_service.cc:1467] Processing CreateTablet for tablet 247b67f292464f82be23c8cc6f011ec9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=07c774bb2e964b3a85a8e539d2cc1b8d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 05:09:49.109421 16273 tablet_service.cc:1467] Processing CreateTablet for tablet 247b67f292464f82be23c8cc6f011ec9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=07c774bb2e964b3a85a8e539d2cc1b8d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 05:09:49.109833 16421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 247b67f292464f82be23c8cc6f011ec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:49.109834 16123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 247b67f292464f82be23c8cc6f011ec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:49.109834 16273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 247b67f292464f82be23c8cc6f011ec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:49.112512 16519 tablet_bootstrap.cc:492] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250: Bootstrap starting.
W20260807 05:09:49.112627 16344 tablet.cc:2369] T 48a967eacd564edfa93c896b309f4f4a P 196776ad987c4d29afccd36feb109f94: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:09:49.113257 16520 tablet_bootstrap.cc:492] T 247b67f292464f82be23c8cc6f011ec9 P 52f3e3469f5d4c1cbe62d8084af37273: Bootstrap starting.
I20260807 05:09:49.113355 16519 tablet_bootstrap.cc:654] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:49.113910 16521 tablet_bootstrap.cc:492] T 247b67f292464f82be23c8cc6f011ec9 P 196776ad987c4d29afccd36feb109f94: Bootstrap starting.
I20260807 05:09:49.114141 16520 tablet_bootstrap.cc:654] T 247b67f292464f82be23c8cc6f011ec9 P 52f3e3469f5d4c1cbe62d8084af37273: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:49.114814 16521 tablet_bootstrap.cc:654] T 247b67f292464f82be23c8cc6f011ec9 P 196776ad987c4d29afccd36feb109f94: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:49.115411 16520 tablet_bootstrap.cc:492] T 247b67f292464f82be23c8cc6f011ec9 P 52f3e3469f5d4c1cbe62d8084af37273: No bootstrap required, opened a new log
I20260807 05:09:49.115499 16520 ts_tablet_manager.cc:1397] T 247b67f292464f82be23c8cc6f011ec9 P 52f3e3469f5d4c1cbe62d8084af37273: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:49.115681 16519 tablet_bootstrap.cc:492] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250: No bootstrap required, opened a new log
I20260807 05:09:49.115762 16519 ts_tablet_manager.cc:1397] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 05:09:49.115916 16520 raft_consensus.cc:348] T 247b67f292464f82be23c8cc6f011ec9 P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.116019 16520 raft_consensus.cc:374] T 247b67f292464f82be23c8cc6f011ec9 P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:49.116055 16520 raft_consensus.cc:729] T 247b67f292464f82be23c8cc6f011ec9 P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52f3e3469f5d4c1cbe62d8084af37273, State: Initialized, Role: FOLLOWER
I20260807 05:09:49.116132 16519 raft_consensus.cc:348] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.116171 16520 consensus_queue.cc:260] T 247b67f292464f82be23c8cc6f011ec9 P 52f3e3469f5d4c1cbe62d8084af37273 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.116240 16519 raft_consensus.cc:374] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:49.116267 16519 raft_consensus.cc:729] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fb9a84dbaa84054b1607e6759c47250, State: Initialized, Role: FOLLOWER
I20260807 05:09:49.116385 16519 consensus_queue.cc:260] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.116426 16520 ts_tablet_manager.cc:1428] T 247b67f292464f82be23c8cc6f011ec9 P 52f3e3469f5d4c1cbe62d8084af37273: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:09:49.116636 16519 ts_tablet_manager.cc:1428] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:09:49.116679 16537 tablet_replica.cc:439] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } } }
I20260807 05:09:49.116863 16537 tablet_replica.cc:442] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:09:49.117786 16521 tablet_bootstrap.cc:492] T 247b67f292464f82be23c8cc6f011ec9 P 196776ad987c4d29afccd36feb109f94: No bootstrap required, opened a new log
I20260807 05:09:49.117892 16521 ts_tablet_manager.cc:1397] T 247b67f292464f82be23c8cc6f011ec9 P 196776ad987c4d29afccd36feb109f94: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260807 05:09:49.118304 16521 raft_consensus.cc:348] T 247b67f292464f82be23c8cc6f011ec9 P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.118413 16521 raft_consensus.cc:374] T 247b67f292464f82be23c8cc6f011ec9 P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:49.118440 16521 raft_consensus.cc:729] T 247b67f292464f82be23c8cc6f011ec9 P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196776ad987c4d29afccd36feb109f94, State: Initialized, Role: FOLLOWER
I20260807 05:09:49.118564 16521 consensus_queue.cc:260] T 247b67f292464f82be23c8cc6f011ec9 P 196776ad987c4d29afccd36feb109f94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.118786 16521 ts_tablet_manager.cc:1428] T 247b67f292464f82be23c8cc6f011ec9 P 196776ad987c4d29afccd36feb109f94: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:09:49.118791 16527 tablet_replica.cc:439] T 247b67f292464f82be23c8cc6f011ec9 P 52f3e3469f5d4c1cbe62d8084af37273: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } } }
I20260807 05:09:49.118927 16527 tablet_replica.cc:442] T 247b67f292464f82be23c8cc6f011ec9 P 52f3e3469f5d4c1cbe62d8084af37273: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:09:49.119031 16526 tablet_replica.cc:439] T 247b67f292464f82be23c8cc6f011ec9 P 196776ad987c4d29afccd36feb109f94: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } } }
I20260807 05:09:49.119158 16526 tablet_replica.cc:442] T 247b67f292464f82be23c8cc6f011ec9 P 196776ad987c4d29afccd36feb109f94: This TxnStatusTablet replica's current role is: FOLLOWER
W20260807 05:09:49.151585 16191 tablet.cc:2369] T 247b67f292464f82be23c8cc6f011ec9 P 52f3e3469f5d4c1cbe62d8084af37273: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:09:49.388438 16537 raft_consensus.cc:482] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:09:49.388589 16537 raft_consensus.cc:504] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.389001 16537 leader_election.cc:288] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52f3e3469f5d4c1cbe62d8084af37273 (127.8.16.65:39621), 196776ad987c4d29afccd36feb109f94 (127.8.16.66:33539)
I20260807 05:09:49.389503 16295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "247b67f292464f82be23c8cc6f011ec9" candidate_uuid: "2fb9a84dbaa84054b1607e6759c47250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196776ad987c4d29afccd36feb109f94" is_pre_election: true
I20260807 05:09:49.389504 16143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "247b67f292464f82be23c8cc6f011ec9" candidate_uuid: "2fb9a84dbaa84054b1607e6759c47250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52f3e3469f5d4c1cbe62d8084af37273" is_pre_election: true
I20260807 05:09:49.389657 16295 raft_consensus.cc:2454] T 247b67f292464f82be23c8cc6f011ec9 P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fb9a84dbaa84054b1607e6759c47250 in term 0.
I20260807 05:09:49.389678 16143 raft_consensus.cc:2454] T 247b67f292464f82be23c8cc6f011ec9 P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fb9a84dbaa84054b1607e6759c47250 in term 0.
I20260807 05:09:49.389966 16371 leader_election.cc:302] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196776ad987c4d29afccd36feb109f94, 2fb9a84dbaa84054b1607e6759c47250; no voters:
I20260807 05:09:49.390169 16537 raft_consensus.cc:2781] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:09:49.390228 16537 raft_consensus.cc:482] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:09:49.390255 16537 raft_consensus.cc:3037] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:49.391393 16537 raft_consensus.cc:504] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.391719 16537 leader_election.cc:288] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250 [CANDIDATE]: Term 1 election: Requested vote from peers 52f3e3469f5d4c1cbe62d8084af37273 (127.8.16.65:39621), 196776ad987c4d29afccd36feb109f94 (127.8.16.66:33539)
I20260807 05:09:49.392066 16295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "247b67f292464f82be23c8cc6f011ec9" candidate_uuid: "2fb9a84dbaa84054b1607e6759c47250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196776ad987c4d29afccd36feb109f94"
I20260807 05:09:49.392081 16143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "247b67f292464f82be23c8cc6f011ec9" candidate_uuid: "2fb9a84dbaa84054b1607e6759c47250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52f3e3469f5d4c1cbe62d8084af37273"
I20260807 05:09:49.392181 16143 raft_consensus.cc:3037] T 247b67f292464f82be23c8cc6f011ec9 P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:49.392174 16295 raft_consensus.cc:3037] T 247b67f292464f82be23c8cc6f011ec9 P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:49.393074 16143 raft_consensus.cc:2454] T 247b67f292464f82be23c8cc6f011ec9 P 52f3e3469f5d4c1cbe62d8084af37273 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fb9a84dbaa84054b1607e6759c47250 in term 1.
I20260807 05:09:49.393110 16295 raft_consensus.cc:2454] T 247b67f292464f82be23c8cc6f011ec9 P 196776ad987c4d29afccd36feb109f94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fb9a84dbaa84054b1607e6759c47250 in term 1.
I20260807 05:09:49.393325 16373 leader_election.cc:302] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fb9a84dbaa84054b1607e6759c47250, 52f3e3469f5d4c1cbe62d8084af37273; no voters:
I20260807 05:09:49.393519 16537 raft_consensus.cc:2781] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:49.393611 16537 raft_consensus.cc:686] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250 [term 1 LEADER]: Becoming Leader. State: Replica: 2fb9a84dbaa84054b1607e6759c47250, State: Running, Role: LEADER
I20260807 05:09:49.393748 16537 consensus_queue.cc:237] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.394335 16525 tablet_replica.cc:439] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250: TxnStatusTablet state changed. Reason: New leader 2fb9a84dbaa84054b1607e6759c47250. Latest consensus state: current_term: 1 leader_uuid: "2fb9a84dbaa84054b1607e6759c47250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } } }
I20260807 05:09:49.394435 16525 tablet_replica.cc:442] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250: This TxnStatusTablet replica's current role is: LEADER
I20260807 05:09:49.394711 16563 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 05:09:49.394867 16563 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 05:09:49.395422 15978 catalog_manager.cc:5495] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2fb9a84dbaa84054b1607e6759c47250 (127.8.16.67). New cstate: current_term: 1 leader_uuid: "2fb9a84dbaa84054b1607e6759c47250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } health_report { overall_health: HEALTHY } } }
I20260807 05:09:49.413969 16558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:49.410703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:42769 (local address 127.0.0.1:58206)
0807 05:09:49.410923 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:49.410960 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:49.411072 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:49.411348 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:49.411351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:49.411360 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 05:09:49.411465 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:49.411471 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.412170 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.412174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:49.412789 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:49.412797 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.413230 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.413233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:49.413327 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:49.413373 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 05:09:49.413662 (+ 289us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 05:09:49.413685 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:09:49.413745 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 05:09:49.413853 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260807 05:09:49.413967 16517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:49.410803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58206 (local address 127.8.16.67:42769)
0807 05:09:49.410934 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:49.410939 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:49.411007 (+ 68us) server_negotiation.cc:407] Connection header received
0807 05:09:49.411199 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:49.411203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:49.411256 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:49.411319 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:49.411566 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.412067 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.412906 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.413128 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.413223 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:49.413502 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:49.413587 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:49.413648 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:49.413768 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 05:09:49.413821 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 05:09:49.416479 16143 raft_consensus.cc:1261] T 247b67f292464f82be23c8cc6f011ec9 P 52f3e3469f5d4c1cbe62d8084af37273 [term 1 FOLLOWER]: Refusing update from remote peer 2fb9a84dbaa84054b1607e6759c47250: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:49.416891 16295 raft_consensus.cc:1261] T 247b67f292464f82be23c8cc6f011ec9 P 196776ad987c4d29afccd36feb109f94 [term 1 FOLLOWER]: Refusing update from remote peer 2fb9a84dbaa84054b1607e6759c47250: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:49.417313 16537 consensus_queue.cc:1035] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:49.417618 16537 consensus_queue.cc:1035] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:49.419106 16527 tablet_replica.cc:439] T 247b67f292464f82be23c8cc6f011ec9 P 52f3e3469f5d4c1cbe62d8084af37273: TxnStatusTablet state changed. Reason: New leader 2fb9a84dbaa84054b1607e6759c47250. Latest consensus state: current_term: 1 leader_uuid: "2fb9a84dbaa84054b1607e6759c47250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } } }
I20260807 05:09:49.419224 16527 tablet_replica.cc:442] T 247b67f292464f82be23c8cc6f011ec9 P 52f3e3469f5d4c1cbe62d8084af37273: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:09:49.419293 16526 tablet_replica.cc:439] T 247b67f292464f82be23c8cc6f011ec9 P 196776ad987c4d29afccd36feb109f94: TxnStatusTablet state changed. Reason: New leader 2fb9a84dbaa84054b1607e6759c47250. Latest consensus state: current_term: 1 leader_uuid: "2fb9a84dbaa84054b1607e6759c47250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } } }
I20260807 05:09:49.419425 16526 tablet_replica.cc:442] T 247b67f292464f82be23c8cc6f011ec9 P 196776ad987c4d29afccd36feb109f94: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:09:49.420689 16525 tablet_replica.cc:439] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2fb9a84dbaa84054b1607e6759c47250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } } }
I20260807 05:09:49.420794 16525 tablet_replica.cc:442] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250: This TxnStatusTablet replica's current role is: LEADER
I20260807 05:09:49.420898 16532 mvcc.cc:205] Tried to move back new op lower bound from 7315781179045617664 to 7315781178957979648. Current Snapshot: MvccSnapshot[applied={T|T < 7315781179045617664}]
I20260807 05:09:49.421999 16526 tablet_replica.cc:439] T 247b67f292464f82be23c8cc6f011ec9 P 196776ad987c4d29afccd36feb109f94: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2fb9a84dbaa84054b1607e6759c47250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } } }
I20260807 05:09:49.422104 16526 tablet_replica.cc:442] T 247b67f292464f82be23c8cc6f011ec9 P 196776ad987c4d29afccd36feb109f94: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:09:49.424850 16529 tablet_replica.cc:439] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2fb9a84dbaa84054b1607e6759c47250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } } }
I20260807 05:09:49.424974 16529 tablet_replica.cc:442] T 247b67f292464f82be23c8cc6f011ec9 P 2fb9a84dbaa84054b1607e6759c47250: This TxnStatusTablet replica's current role is: LEADER
I20260807 05:09:49.425823 16527 tablet_replica.cc:439] T 247b67f292464f82be23c8cc6f011ec9 P 52f3e3469f5d4c1cbe62d8084af37273: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2fb9a84dbaa84054b1607e6759c47250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } } }
I20260807 05:09:49.425935 16527 tablet_replica.cc:442] T 247b67f292464f82be23c8cc6f011ec9 P 52f3e3469f5d4c1cbe62d8084af37273: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:09:49.433314 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:49.426294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49734 (local address 127.8.16.65:39621)
0807 05:09:49.429621 (+ 3327us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:49.429625 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:49.429646 (+ 21us) server_negotiation.cc:407] Connection header received
0807 05:09:49.429724 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:49.429727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:49.429787 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:49.429908 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:49.430151 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.430940 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.431692 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.432036 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.432131 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:49.432339 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:49.432890 (+ 551us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:49.433003 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:49.433101 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 05:09:49.433161 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3253,"thread_start_us":63,"threads_started":1}
I20260807 05:09:49.443186 16517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:49.440203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:57651 (local address 127.8.16.67:42769)
0807 05:09:49.440322 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:49.440327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:49.440457 (+ 130us) server_negotiation.cc:407] Connection header received
0807 05:09:49.440587 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:49.440598 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:49.440640 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:49.440695 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:49.440912 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.441403 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.442236 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.442445 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.442528 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:49.442782 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:49.442845 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:49.442901 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:49.443003 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 05:09:49.443048 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 05:09:49.443229 16571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:49.440113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:42769 (local address 127.8.16.65:57651)
0807 05:09:49.440384 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:49.440396 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:49.440486 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:49.440725 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:49.440728 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:49.440735 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 05:09:49.440826 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:49.440830 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.441527 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.441530 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:49.442129 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:49.442137 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.442546 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.442549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:49.442623 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:49.442662 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 05:09:49.442923 (+ 261us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 05:09:49.442929 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 05:09:49.442986 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 05:09:49.443059 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":97,"threads_started":1}
I20260807 05:09:49.450026 16442 raft_consensus.cc:1261] T 48a967eacd564edfa93c896b309f4f4a P 2fb9a84dbaa84054b1607e6759c47250 [term 1 FOLLOWER]: Refusing update from remote peer 52f3e3469f5d4c1cbe62d8084af37273: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:49.450491 16295 raft_consensus.cc:1261] T 48a967eacd564edfa93c896b309f4f4a P 196776ad987c4d29afccd36feb109f94 [term 1 FOLLOWER]: Refusing update from remote peer 52f3e3469f5d4c1cbe62d8084af37273: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:49.450577 16527 consensus_queue.cc:1035] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:49.450930 16551 consensus_queue.cc:1035] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273 [LEADER]: Connected to new peer: Peer: permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:49.474080 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:49.470300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:46929 (local address 127.8.16.65:39621)
0807 05:09:49.470413 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:49.470418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:49.470580 (+ 162us) server_negotiation.cc:407] Connection header received
0807 05:09:49.470694 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:49.470697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:49.470744 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:49.470802 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:49.471081 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.471866 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.473068 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.473291 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.473367 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:49.473688 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:49.473754 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:49.473804 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:49.473889 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 05:09:49.473944 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 05:09:49.474097 16576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:49.470241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:39621 (local address 127.8.16.67:46929)
0807 05:09:49.470508 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:49.470520 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:49.470616 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:49.470819 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:49.470822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:49.470842 (+ 20us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 05:09:49.470960 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:49.470967 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.471993 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.471996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:49.472961 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:49.472971 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.473390 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.473394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:49.473508 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:49.473559 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 05:09:49.473820 (+ 261us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 05:09:49.473825 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 05:09:49.473880 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 05:09:49.473960 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":88,"threads_started":1}
I20260807 05:09:49.497773 15979 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:49.497952 15979 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:49.500142 15979 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f: Sending DeleteTablet for 3 replicas of tablet 48a967eacd564edfa93c896b309f4f4a
I20260807 05:09:49.500680 16123 tablet_service.cc:1514] Processing DeleteTablet for tablet 48a967eacd564edfa93c896b309f4f4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:49 UTC) from {username='oryx', principal='oryx/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:49732
I20260807 05:09:49.500820 16273 tablet_service.cc:1514] Processing DeleteTablet for tablet 48a967eacd564edfa93c896b309f4f4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:49 UTC) from {username='oryx', principal='oryx/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:35250
I20260807 05:09:49.501113 16579 tablet_replica.cc:330] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273: stopping tablet replica
I20260807 05:09:49.501288 16579 raft_consensus.cc:2229] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:09:49.501367 16421 tablet_service.cc:1514] Processing DeleteTablet for tablet 48a967eacd564edfa93c896b309f4f4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:49 UTC) from {username='oryx', principal='oryx/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:58194
I20260807 05:09:49.501722 16579 raft_consensus.cc:2258] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:49.501992 16581 tablet_replica.cc:330] T 48a967eacd564edfa93c896b309f4f4a P 2fb9a84dbaa84054b1607e6759c47250: stopping tablet replica
I20260807 05:09:49.502113 16581 raft_consensus.cc:2229] T 48a967eacd564edfa93c896b309f4f4a P 2fb9a84dbaa84054b1607e6759c47250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:49.502228 16581 raft_consensus.cc:2258] T 48a967eacd564edfa93c896b309f4f4a P 2fb9a84dbaa84054b1607e6759c47250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:49.502444 16580 tablet_replica.cc:330] T 48a967eacd564edfa93c896b309f4f4a P 196776ad987c4d29afccd36feb109f94: stopping tablet replica
I20260807 05:09:49.502558 16580 raft_consensus.cc:2229] T 48a967eacd564edfa93c896b309f4f4a P 196776ad987c4d29afccd36feb109f94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:49.502674 16580 raft_consensus.cc:2258] T 48a967eacd564edfa93c896b309f4f4a P 196776ad987c4d29afccd36feb109f94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:49.503165 16581 ts_tablet_manager.cc:1905] T 48a967eacd564edfa93c896b309f4f4a P 2fb9a84dbaa84054b1607e6759c47250: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:49.503177 16579 ts_tablet_manager.cc:1905] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:49.506047 16581 ts_tablet_manager.cc:1918] T 48a967eacd564edfa93c896b309f4f4a P 2fb9a84dbaa84054b1607e6759c47250: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:09:49.506145 16581 log.cc:1196] T 48a967eacd564edfa93c896b309f4f4a P 2fb9a84dbaa84054b1607e6759c47250: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/48a967eacd564edfa93c896b309f4f4a
I20260807 05:09:49.506422 16581 ts_tablet_manager.cc:1939] T 48a967eacd564edfa93c896b309f4f4a P 2fb9a84dbaa84054b1607e6759c47250: Deleting consensus metadata
I20260807 05:09:49.506564 16579 ts_tablet_manager.cc:1918] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:09:49.506706 16579 log.cc:1196] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-0/wal/wals/48a967eacd564edfa93c896b309f4f4a
I20260807 05:09:49.507014 16579 ts_tablet_manager.cc:1939] T 48a967eacd564edfa93c896b309f4f4a P 52f3e3469f5d4c1cbe62d8084af37273: Deleting consensus metadata
I20260807 05:09:49.507088 15966 catalog_manager.cc:4841] TS 2fb9a84dbaa84054b1607e6759c47250 (127.8.16.67:42769): tablet 48a967eacd564edfa93c896b309f4f4a (table test-table [id=a27675e86de74ab4a3002f665c36edf0]) successfully deleted
I20260807 05:09:49.507668 16580 ts_tablet_manager.cc:1905] T 48a967eacd564edfa93c896b309f4f4a P 196776ad987c4d29afccd36feb109f94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:49.507755 15965 catalog_manager.cc:4841] TS 52f3e3469f5d4c1cbe62d8084af37273 (127.8.16.65:39621): tablet 48a967eacd564edfa93c896b309f4f4a (table test-table [id=a27675e86de74ab4a3002f665c36edf0]) successfully deleted
I20260807 05:09:49.510509 16580 ts_tablet_manager.cc:1918] T 48a967eacd564edfa93c896b309f4f4a P 196776ad987c4d29afccd36feb109f94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:09:49.510587 16580 log.cc:1196] T 48a967eacd564edfa93c896b309f4f4a P 196776ad987c4d29afccd36feb109f94: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-1/wal/wals/48a967eacd564edfa93c896b309f4f4a
I20260807 05:09:49.510840 16580 ts_tablet_manager.cc:1939] T 48a967eacd564edfa93c896b309f4f4a P 196776ad987c4d29afccd36feb109f94: Deleting consensus metadata
I20260807 05:09:49.511469 15963 catalog_manager.cc:4841] TS 196776ad987c4d29afccd36feb109f94 (127.8.16.66:33539): tablet 48a967eacd564edfa93c896b309f4f4a (table test-table [id=a27675e86de74ab4a3002f665c36edf0]) successfully deleted
Aug 07 05:09:49 dist-test-slave-49h3 krb5kdc[15932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 05:09:49 dist-test-slave-49h3 krb5kdc[15932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:49 dist-test-slave-49h3 krb5kdc[15932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 05:09:49.538673 16032 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:49.530996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43206 (local address 127.8.16.126:38273)
0807 05:09:49.531121 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:49.531126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:49.531406 (+ 280us) server_negotiation.cc:407] Connection header received
0807 05:09:49.531540 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:49.531543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:49.531604 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:49.531709 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:49.532539 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.533382 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.534678 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.535007 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.535104 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:49.538530 (+ 3426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43206: BlockingRecv error: recv got EOF from 127.0.0.1:43206 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 07 05:09:49 dist-test-slave-49h3 krb5kdc[15932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM
I20260807 05:09:49.550050 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:49.540468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43222 (local address 127.8.16.126:38273)
0807 05:09:49.540571 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:49.540575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:49.540778 (+ 203us) server_negotiation.cc:407] Connection header received
0807 05:09:49.540875 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:49.540878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:49.540921 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:49.540999 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:49.541737 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.542514 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.543579 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.543918 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.544009 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:49.546759 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:49.546762 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:49.546765 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:49.546794 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:49.548398 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:49.548840 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:49.548843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:49.548845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:49.548923 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:49.549159 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:49.549161 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:49.549163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:49.549358 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:49.549504 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:49.549734 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 05:09:49.549843 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 05:09:49.552775 15979 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43222:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:09:49.553233 15979 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:09:49.558207 16123 tablet_service.cc:1467] Processing CreateTablet for tablet c1f49504d8b149a7b59014652c15fb5c (DEFAULT_TABLE table=test-table [id=ee639c6e4306441f94c1d573f6315dc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:49.558216 16273 tablet_service.cc:1467] Processing CreateTablet for tablet c1f49504d8b149a7b59014652c15fb5c (DEFAULT_TABLE table=test-table [id=ee639c6e4306441f94c1d573f6315dc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:49.558347 16421 tablet_service.cc:1467] Processing CreateTablet for tablet c1f49504d8b149a7b59014652c15fb5c (DEFAULT_TABLE table=test-table [id=ee639c6e4306441f94c1d573f6315dc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:49.558513 16273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1f49504d8b149a7b59014652c15fb5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:49.558513 16123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1f49504d8b149a7b59014652c15fb5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:49.558574 16421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1f49504d8b149a7b59014652c15fb5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:49.560829 16521 tablet_bootstrap.cc:492] T c1f49504d8b149a7b59014652c15fb5c P 196776ad987c4d29afccd36feb109f94: Bootstrap starting.
I20260807 05:09:49.560847 16519 tablet_bootstrap.cc:492] T c1f49504d8b149a7b59014652c15fb5c P 2fb9a84dbaa84054b1607e6759c47250: Bootstrap starting.
I20260807 05:09:49.560847 16520 tablet_bootstrap.cc:492] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273: Bootstrap starting.
I20260807 05:09:49.561685 16519 tablet_bootstrap.cc:654] T c1f49504d8b149a7b59014652c15fb5c P 2fb9a84dbaa84054b1607e6759c47250: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:49.561685 16520 tablet_bootstrap.cc:654] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:49.561734 16521 tablet_bootstrap.cc:654] T c1f49504d8b149a7b59014652c15fb5c P 196776ad987c4d29afccd36feb109f94: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:49.563071 16520 tablet_bootstrap.cc:492] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273: No bootstrap required, opened a new log
I20260807 05:09:49.563165 16520 ts_tablet_manager.cc:1397] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:49.563512 16521 tablet_bootstrap.cc:492] T c1f49504d8b149a7b59014652c15fb5c P 196776ad987c4d29afccd36feb109f94: No bootstrap required, opened a new log
I20260807 05:09:49.563601 16521 ts_tablet_manager.cc:1397] T c1f49504d8b149a7b59014652c15fb5c P 196776ad987c4d29afccd36feb109f94: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 05:09:49.563628 16520 raft_consensus.cc:348] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.563714 16519 tablet_bootstrap.cc:492] T c1f49504d8b149a7b59014652c15fb5c P 2fb9a84dbaa84054b1607e6759c47250: No bootstrap required, opened a new log
I20260807 05:09:49.563750 16520 raft_consensus.cc:374] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:49.563782 16519 ts_tablet_manager.cc:1397] T c1f49504d8b149a7b59014652c15fb5c P 2fb9a84dbaa84054b1607e6759c47250: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 05:09:49.563781 16520 raft_consensus.cc:729] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52f3e3469f5d4c1cbe62d8084af37273, State: Initialized, Role: FOLLOWER
I20260807 05:09:49.563899 16520 consensus_queue.cc:260] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.564046 16521 raft_consensus.cc:348] T c1f49504d8b149a7b59014652c15fb5c P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.564136 16520 ts_tablet_manager.cc:1428] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:09:49.564144 16521 raft_consensus.cc:374] T c1f49504d8b149a7b59014652c15fb5c P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:49.564177 16521 raft_consensus.cc:729] T c1f49504d8b149a7b59014652c15fb5c P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196776ad987c4d29afccd36feb109f94, State: Initialized, Role: FOLLOWER
I20260807 05:09:49.564145 16519 raft_consensus.cc:348] T c1f49504d8b149a7b59014652c15fb5c P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.564244 16519 raft_consensus.cc:374] T c1f49504d8b149a7b59014652c15fb5c P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:49.564275 16519 raft_consensus.cc:729] T c1f49504d8b149a7b59014652c15fb5c P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fb9a84dbaa84054b1607e6759c47250, State: Initialized, Role: FOLLOWER
I20260807 05:09:49.564270 16527 raft_consensus.cc:482] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:09:49.564301 16521 consensus_queue.cc:260] T c1f49504d8b149a7b59014652c15fb5c P 196776ad987c4d29afccd36feb109f94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.564328 16527 raft_consensus.cc:504] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.564388 16519 consensus_queue.cc:260] T c1f49504d8b149a7b59014652c15fb5c P 2fb9a84dbaa84054b1607e6759c47250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.564555 16521 ts_tablet_manager.cc:1428] T c1f49504d8b149a7b59014652c15fb5c P 196776ad987c4d29afccd36feb109f94: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:09:49.564616 16519 ts_tablet_manager.cc:1428] T c1f49504d8b149a7b59014652c15fb5c P 2fb9a84dbaa84054b1607e6759c47250: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 05:09:49.564664 16527 leader_election.cc:288] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 196776ad987c4d29afccd36feb109f94 (127.8.16.66:33539), 2fb9a84dbaa84054b1607e6759c47250 (127.8.16.67:42769)
I20260807 05:09:49.565809 16295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1f49504d8b149a7b59014652c15fb5c" candidate_uuid: "52f3e3469f5d4c1cbe62d8084af37273" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196776ad987c4d29afccd36feb109f94" is_pre_election: true
I20260807 05:09:49.565879 16442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1f49504d8b149a7b59014652c15fb5c" candidate_uuid: "52f3e3469f5d4c1cbe62d8084af37273" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fb9a84dbaa84054b1607e6759c47250" is_pre_election: true
I20260807 05:09:49.565932 16295 raft_consensus.cc:2454] T c1f49504d8b149a7b59014652c15fb5c P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52f3e3469f5d4c1cbe62d8084af37273 in term 0.
I20260807 05:09:49.565976 16442 raft_consensus.cc:2454] T c1f49504d8b149a7b59014652c15fb5c P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52f3e3469f5d4c1cbe62d8084af37273 in term 0.
I20260807 05:09:49.566247 16072 leader_election.cc:302] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196776ad987c4d29afccd36feb109f94, 52f3e3469f5d4c1cbe62d8084af37273; no voters:
I20260807 05:09:49.566401 16551 raft_consensus.cc:2781] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:09:49.566448 16551 raft_consensus.cc:482] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:09:49.566485 16551 raft_consensus.cc:3037] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:49.567588 16551 raft_consensus.cc:504] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.567939 16551 leader_election.cc:288] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273 [CANDIDATE]: Term 1 election: Requested vote from peers 196776ad987c4d29afccd36feb109f94 (127.8.16.66:33539), 2fb9a84dbaa84054b1607e6759c47250 (127.8.16.67:42769)
I20260807 05:09:49.568332 16295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1f49504d8b149a7b59014652c15fb5c" candidate_uuid: "52f3e3469f5d4c1cbe62d8084af37273" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196776ad987c4d29afccd36feb109f94"
I20260807 05:09:49.568440 16295 raft_consensus.cc:3037] T c1f49504d8b149a7b59014652c15fb5c P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:49.568660 16442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1f49504d8b149a7b59014652c15fb5c" candidate_uuid: "52f3e3469f5d4c1cbe62d8084af37273" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fb9a84dbaa84054b1607e6759c47250"
I20260807 05:09:49.568768 16442 raft_consensus.cc:3037] T c1f49504d8b149a7b59014652c15fb5c P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:49.569409 16295 raft_consensus.cc:2454] T c1f49504d8b149a7b59014652c15fb5c P 196776ad987c4d29afccd36feb109f94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52f3e3469f5d4c1cbe62d8084af37273 in term 1.
I20260807 05:09:49.569803 16072 leader_election.cc:302] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196776ad987c4d29afccd36feb109f94, 52f3e3469f5d4c1cbe62d8084af37273; no voters:
I20260807 05:09:49.569803 16442 raft_consensus.cc:2454] T c1f49504d8b149a7b59014652c15fb5c P 2fb9a84dbaa84054b1607e6759c47250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52f3e3469f5d4c1cbe62d8084af37273 in term 1.
I20260807 05:09:49.569970 16551 raft_consensus.cc:2781] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:49.570046 16551 raft_consensus.cc:686] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273 [term 1 LEADER]: Becoming Leader. State: Replica: 52f3e3469f5d4c1cbe62d8084af37273, State: Running, Role: LEADER
I20260807 05:09:49.570183 16551 consensus_queue.cc:237] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.571530 15978 catalog_manager.cc:5495] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52f3e3469f5d4c1cbe62d8084af37273 (127.8.16.65). New cstate: current_term: 1 leader_uuid: "52f3e3469f5d4c1cbe62d8084af37273" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } health_report { overall_health: UNKNOWN } } }
I20260807 05:09:49.580005 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:49.576841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49736 (local address 127.8.16.65:39621)
0807 05:09:49.576951 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:49.576954 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:49.576966 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:09:49.577069 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:49.577071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:49.577109 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:49.577167 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:49.577394 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.577935 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.579096 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.579314 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.579388 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:49.579656 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:49.579718 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:49.579762 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:49.579848 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 05:09:49.579889 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 05:09:49.581794 16295 raft_consensus.cc:1261] T c1f49504d8b149a7b59014652c15fb5c P 196776ad987c4d29afccd36feb109f94 [term 1 FOLLOWER]: Refusing update from remote peer 52f3e3469f5d4c1cbe62d8084af37273: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:49.581792 16442 raft_consensus.cc:1261] T c1f49504d8b149a7b59014652c15fb5c P 2fb9a84dbaa84054b1607e6759c47250 [term 1 FOLLOWER]: Refusing update from remote peer 52f3e3469f5d4c1cbe62d8084af37273: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:49.582162 16551 consensus_queue.cc:1035] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:49.582309 16527 consensus_queue.cc:1035] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273 [LEADER]: Connected to new peer: Peer: permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:49.592250 15978 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:49.592370 15978 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:49.594129 15978 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f: Sending DeleteTablet for 3 replicas of tablet c1f49504d8b149a7b59014652c15fb5c
I20260807 05:09:49.594671 16123 tablet_service.cc:1514] Processing DeleteTablet for tablet c1f49504d8b149a7b59014652c15fb5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:49 UTC) from {username='oryx', principal='oryx/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:49732
I20260807 05:09:49.594780 16273 tablet_service.cc:1514] Processing DeleteTablet for tablet c1f49504d8b149a7b59014652c15fb5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:49 UTC) from {username='oryx', principal='oryx/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:35250
I20260807 05:09:49.594842 16579 tablet_replica.cc:330] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273: stopping tablet replica
I20260807 05:09:49.594944 16579 raft_consensus.cc:2229] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:09:49.594957 16580 tablet_replica.cc:330] T c1f49504d8b149a7b59014652c15fb5c P 196776ad987c4d29afccd36feb109f94: stopping tablet replica
I20260807 05:09:49.595069 16580 raft_consensus.cc:2229] T c1f49504d8b149a7b59014652c15fb5c P 196776ad987c4d29afccd36feb109f94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:49.595114 16579 raft_consensus.cc:2258] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:49.595192 16580 raft_consensus.cc:2258] T c1f49504d8b149a7b59014652c15fb5c P 196776ad987c4d29afccd36feb109f94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:49.595621 16421 tablet_service.cc:1514] Processing DeleteTablet for tablet c1f49504d8b149a7b59014652c15fb5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:49 UTC) from {username='oryx', principal='oryx/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:58194
I20260807 05:09:49.595806 16581 tablet_replica.cc:330] T c1f49504d8b149a7b59014652c15fb5c P 2fb9a84dbaa84054b1607e6759c47250: stopping tablet replica
I20260807 05:09:49.595810 16580 ts_tablet_manager.cc:1905] T c1f49504d8b149a7b59014652c15fb5c P 196776ad987c4d29afccd36feb109f94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:49.595916 16581 raft_consensus.cc:2229] T c1f49504d8b149a7b59014652c15fb5c P 2fb9a84dbaa84054b1607e6759c47250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:49.595904 15978 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43222:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 05:09:49.596030 16581 raft_consensus.cc:2258] T c1f49504d8b149a7b59014652c15fb5c P 2fb9a84dbaa84054b1607e6759c47250 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 05:09:49.596295 15978 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:09:49.596556 16579 ts_tablet_manager.cc:1905] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:49.598305 16580 ts_tablet_manager.cc:1918] T c1f49504d8b149a7b59014652c15fb5c P 196776ad987c4d29afccd36feb109f94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:09:49.598387 16580 log.cc:1196] T c1f49504d8b149a7b59014652c15fb5c P 196776ad987c4d29afccd36feb109f94: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-1/wal/wals/c1f49504d8b149a7b59014652c15fb5c
I20260807 05:09:49.598778 16580 ts_tablet_manager.cc:1939] T c1f49504d8b149a7b59014652c15fb5c P 196776ad987c4d29afccd36feb109f94: Deleting consensus metadata
I20260807 05:09:49.598929 16579 ts_tablet_manager.cc:1918] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:09:49.599001 16579 log.cc:1196] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-0/wal/wals/c1f49504d8b149a7b59014652c15fb5c
I20260807 05:09:49.599221 16579 ts_tablet_manager.cc:1939] T c1f49504d8b149a7b59014652c15fb5c P 52f3e3469f5d4c1cbe62d8084af37273: Deleting consensus metadata
I20260807 05:09:49.599406 15963 catalog_manager.cc:4841] TS 196776ad987c4d29afccd36feb109f94 (127.8.16.66:33539): tablet c1f49504d8b149a7b59014652c15fb5c (table test-table [id=ee639c6e4306441f94c1d573f6315dc6]) successfully deleted
I20260807 05:09:49.599820 15965 catalog_manager.cc:4841] TS 52f3e3469f5d4c1cbe62d8084af37273 (127.8.16.65:39621): tablet c1f49504d8b149a7b59014652c15fb5c (table test-table [id=ee639c6e4306441f94c1d573f6315dc6]) successfully deleted
I20260807 05:09:49.600757 16581 ts_tablet_manager.cc:1905] T c1f49504d8b149a7b59014652c15fb5c P 2fb9a84dbaa84054b1607e6759c47250: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:49.603157 16581 ts_tablet_manager.cc:1918] T c1f49504d8b149a7b59014652c15fb5c P 2fb9a84dbaa84054b1607e6759c47250: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:09:49.603140 16123 tablet_service.cc:1467] Processing CreateTablet for tablet bd713536d4a44f9793a993d7ee377737 (DEFAULT_TABLE table=test-table [id=f34f32a8656a4030884690eaf2f20f2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:49.603233 16581 log.cc:1196] T c1f49504d8b149a7b59014652c15fb5c P 2fb9a84dbaa84054b1607e6759c47250: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/c1f49504d8b149a7b59014652c15fb5c
I20260807 05:09:49.603319 16273 tablet_service.cc:1467] Processing CreateTablet for tablet bd713536d4a44f9793a993d7ee377737 (DEFAULT_TABLE table=test-table [id=f34f32a8656a4030884690eaf2f20f2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:49.603401 16123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd713536d4a44f9793a993d7ee377737. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:49.603453 16581 ts_tablet_manager.cc:1939] T c1f49504d8b149a7b59014652c15fb5c P 2fb9a84dbaa84054b1607e6759c47250: Deleting consensus metadata
I20260807 05:09:49.603535 16273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd713536d4a44f9793a993d7ee377737. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:49.605558 16521 tablet_bootstrap.cc:492] T bd713536d4a44f9793a993d7ee377737 P 196776ad987c4d29afccd36feb109f94: Bootstrap starting.
I20260807 05:09:49.605628 15966 catalog_manager.cc:4841] TS 2fb9a84dbaa84054b1607e6759c47250 (127.8.16.67:42769): tablet c1f49504d8b149a7b59014652c15fb5c (table test-table [id=ee639c6e4306441f94c1d573f6315dc6]) successfully deleted
I20260807 05:09:49.606060 16421 tablet_service.cc:1467] Processing CreateTablet for tablet bd713536d4a44f9793a993d7ee377737 (DEFAULT_TABLE table=test-table [id=f34f32a8656a4030884690eaf2f20f2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:09:49.606331 16421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd713536d4a44f9793a993d7ee377737. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:49.606904 16521 tablet_bootstrap.cc:654] T bd713536d4a44f9793a993d7ee377737 P 196776ad987c4d29afccd36feb109f94: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:49.607566 16520 tablet_bootstrap.cc:492] T bd713536d4a44f9793a993d7ee377737 P 52f3e3469f5d4c1cbe62d8084af37273: Bootstrap starting.
I20260807 05:09:49.608354 16519 tablet_bootstrap.cc:492] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250: Bootstrap starting.
I20260807 05:09:49.608377 16520 tablet_bootstrap.cc:654] T bd713536d4a44f9793a993d7ee377737 P 52f3e3469f5d4c1cbe62d8084af37273: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:49.609128 16519 tablet_bootstrap.cc:654] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:49.610137 16519 tablet_bootstrap.cc:492] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250: No bootstrap required, opened a new log
I20260807 05:09:49.610216 16519 ts_tablet_manager.cc:1397] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:49.610559 16519 raft_consensus.cc:348] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.610654 16519 raft_consensus.cc:374] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:49.610683 16519 raft_consensus.cc:729] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fb9a84dbaa84054b1607e6759c47250, State: Initialized, Role: FOLLOWER
I20260807 05:09:49.610828 16519 consensus_queue.cc:260] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.611074 16519 ts_tablet_manager.cc:1428] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:09:49.613917 16521 tablet_bootstrap.cc:492] T bd713536d4a44f9793a993d7ee377737 P 196776ad987c4d29afccd36feb109f94: No bootstrap required, opened a new log
I20260807 05:09:49.614012 16521 ts_tablet_manager.cc:1397] T bd713536d4a44f9793a993d7ee377737 P 196776ad987c4d29afccd36feb109f94: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260807 05:09:49.614396 16521 raft_consensus.cc:348] T bd713536d4a44f9793a993d7ee377737 P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.614492 16521 raft_consensus.cc:374] T bd713536d4a44f9793a993d7ee377737 P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:49.614522 16521 raft_consensus.cc:729] T bd713536d4a44f9793a993d7ee377737 P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196776ad987c4d29afccd36feb109f94, State: Initialized, Role: FOLLOWER
I20260807 05:09:49.614646 16521 consensus_queue.cc:260] T bd713536d4a44f9793a993d7ee377737 P 196776ad987c4d29afccd36feb109f94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.614880 16521 ts_tablet_manager.cc:1428] T bd713536d4a44f9793a993d7ee377737 P 196776ad987c4d29afccd36feb109f94: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 05:09:49.615814 16520 tablet_bootstrap.cc:492] T bd713536d4a44f9793a993d7ee377737 P 52f3e3469f5d4c1cbe62d8084af37273: No bootstrap required, opened a new log
I20260807 05:09:49.615912 16520 ts_tablet_manager.cc:1397] T bd713536d4a44f9793a993d7ee377737 P 52f3e3469f5d4c1cbe62d8084af37273: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260807 05:09:49.616714 16520 raft_consensus.cc:348] T bd713536d4a44f9793a993d7ee377737 P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.616837 16520 raft_consensus.cc:374] T bd713536d4a44f9793a993d7ee377737 P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:49.616878 16520 raft_consensus.cc:729] T bd713536d4a44f9793a993d7ee377737 P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52f3e3469f5d4c1cbe62d8084af37273, State: Initialized, Role: FOLLOWER
I20260807 05:09:49.616988 16520 consensus_queue.cc:260] T bd713536d4a44f9793a993d7ee377737 P 52f3e3469f5d4c1cbe62d8084af37273 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.617223 16520 ts_tablet_manager.cc:1428] T bd713536d4a44f9793a993d7ee377737 P 52f3e3469f5d4c1cbe62d8084af37273: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:09:49.776347 16537 raft_consensus.cc:482] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:09:49.776494 16537 raft_consensus.cc:504] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.776912 16537 leader_election.cc:288] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52f3e3469f5d4c1cbe62d8084af37273 (127.8.16.65:39621), 196776ad987c4d29afccd36feb109f94 (127.8.16.66:33539)
I20260807 05:09:49.777473 16143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd713536d4a44f9793a993d7ee377737" candidate_uuid: "2fb9a84dbaa84054b1607e6759c47250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52f3e3469f5d4c1cbe62d8084af37273" is_pre_election: true
I20260807 05:09:49.777467 16295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd713536d4a44f9793a993d7ee377737" candidate_uuid: "2fb9a84dbaa84054b1607e6759c47250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196776ad987c4d29afccd36feb109f94" is_pre_election: true
I20260807 05:09:49.777635 16143 raft_consensus.cc:2454] T bd713536d4a44f9793a993d7ee377737 P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fb9a84dbaa84054b1607e6759c47250 in term 0.
I20260807 05:09:49.777635 16295 raft_consensus.cc:2454] T bd713536d4a44f9793a993d7ee377737 P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fb9a84dbaa84054b1607e6759c47250 in term 0.
I20260807 05:09:49.777930 16371 leader_election.cc:302] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196776ad987c4d29afccd36feb109f94, 2fb9a84dbaa84054b1607e6759c47250; no voters:
I20260807 05:09:49.778103 16537 raft_consensus.cc:2781] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:09:49.778167 16537 raft_consensus.cc:482] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:09:49.778194 16537 raft_consensus.cc:3037] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:49.779201 16537 raft_consensus.cc:504] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.779522 16537 leader_election.cc:288] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250 [CANDIDATE]: Term 1 election: Requested vote from peers 52f3e3469f5d4c1cbe62d8084af37273 (127.8.16.65:39621), 196776ad987c4d29afccd36feb109f94 (127.8.16.66:33539)
I20260807 05:09:49.779852 16143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd713536d4a44f9793a993d7ee377737" candidate_uuid: "2fb9a84dbaa84054b1607e6759c47250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52f3e3469f5d4c1cbe62d8084af37273"
I20260807 05:09:49.779883 16295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd713536d4a44f9793a993d7ee377737" candidate_uuid: "2fb9a84dbaa84054b1607e6759c47250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196776ad987c4d29afccd36feb109f94"
I20260807 05:09:49.779968 16143 raft_consensus.cc:3037] T bd713536d4a44f9793a993d7ee377737 P 52f3e3469f5d4c1cbe62d8084af37273 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:49.779990 16295 raft_consensus.cc:3037] T bd713536d4a44f9793a993d7ee377737 P 196776ad987c4d29afccd36feb109f94 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:49.780821 16295 raft_consensus.cc:2454] T bd713536d4a44f9793a993d7ee377737 P 196776ad987c4d29afccd36feb109f94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fb9a84dbaa84054b1607e6759c47250 in term 1.
I20260807 05:09:49.780884 16143 raft_consensus.cc:2454] T bd713536d4a44f9793a993d7ee377737 P 52f3e3469f5d4c1cbe62d8084af37273 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fb9a84dbaa84054b1607e6759c47250 in term 1.
I20260807 05:09:49.781075 16371 leader_election.cc:302] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196776ad987c4d29afccd36feb109f94, 2fb9a84dbaa84054b1607e6759c47250; no voters:
I20260807 05:09:49.781241 16537 raft_consensus.cc:2781] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:49.781329 16537 raft_consensus.cc:686] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250 [term 1 LEADER]: Becoming Leader. State: Replica: 2fb9a84dbaa84054b1607e6759c47250, State: Running, Role: LEADER
I20260807 05:09:49.781497 16537 consensus_queue.cc:237] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } }
I20260807 05:09:49.783150 15979 catalog_manager.cc:5495] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2fb9a84dbaa84054b1607e6759c47250 (127.8.16.67). New cstate: current_term: 1 leader_uuid: "2fb9a84dbaa84054b1607e6759c47250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fb9a84dbaa84054b1607e6759c47250" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42769 } health_report { overall_health: HEALTHY } } }
I20260807 05:09:49.806257 16517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:49.803561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58208 (local address 127.8.16.67:42769)
0807 05:09:49.803663 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:49.803667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:49.803686 (+ 19us) server_negotiation.cc:407] Connection header received
0807 05:09:49.803819 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:49.803822 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:49.803865 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:49.803923 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:49.804124 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.804621 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.805392 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.805652 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.805737 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:49.805928 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:49.805992 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:49.806038 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:49.806106 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 05:09:49.806146 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 05:09:49.814314 16576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:49.811327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:42769 (local address 127.8.16.67:35171)
0807 05:09:49.811495 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:49.811509 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:49.811596 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:49.811795 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:49.811798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:49.811804 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 05:09:49.811905 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:49.811910 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.812557 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.812560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:49.813141 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:49.813149 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.813596 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.813599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:49.813687 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:49.813712 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 05:09:49.813985 (+ 273us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 05:09:49.813989 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 05:09:49.814052 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 05:09:49.814153 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":19}
I20260807 05:09:49.814314 16517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:49.811417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:35171 (local address 127.8.16.67:42769)
0807 05:09:49.811510 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:49.811514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:49.811554 (+ 40us) server_negotiation.cc:407] Connection header received
0807 05:09:49.811679 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:49.811682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:49.811721 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:49.811779 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:49.811993 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.812461 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.813250 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:49.813495 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:49.813585 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:49.813827 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:49.813907 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:49.813985 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:49.814057 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 05:09:49.814104 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":8,"server-negotiator.queue_time_us":40}
I20260807 05:09:49.819726 16295 raft_consensus.cc:1261] T bd713536d4a44f9793a993d7ee377737 P 196776ad987c4d29afccd36feb109f94 [term 1 FOLLOWER]: Refusing update from remote peer 2fb9a84dbaa84054b1607e6759c47250: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:49.819816 16142 raft_consensus.cc:1261] T bd713536d4a44f9793a993d7ee377737 P 52f3e3469f5d4c1cbe62d8084af37273 [term 1 FOLLOWER]: Refusing update from remote peer 2fb9a84dbaa84054b1607e6759c47250: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:49.820125 16537 consensus_queue.cc:1035] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "196776ad987c4d29afccd36feb109f94" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:49.820322 16529 consensus_queue.cc:1035] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52f3e3469f5d4c1cbe62d8084af37273" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:49.850005 15978 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:49.850175 15978 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:09:49.851966 15978 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e5f85b8e2fb4bb2aeba733336db713f: Sending DeleteTablet for 3 replicas of tablet bd713536d4a44f9793a993d7ee377737
I20260807 05:09:49.852504 16273 tablet_service.cc:1514] Processing DeleteTablet for tablet bd713536d4a44f9793a993d7ee377737 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:49 UTC) from {username='oryx', principal='oryx/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:35250
I20260807 05:09:49.852505 16123 tablet_service.cc:1514] Processing DeleteTablet for tablet bd713536d4a44f9793a993d7ee377737 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:49 UTC) from {username='oryx', principal='oryx/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:49732
I20260807 05:09:49.852690 16579 tablet_replica.cc:330] T bd713536d4a44f9793a993d7ee377737 P 52f3e3469f5d4c1cbe62d8084af37273: stopping tablet replica
I20260807 05:09:49.852699 16420 tablet_service.cc:1514] Processing DeleteTablet for tablet bd713536d4a44f9793a993d7ee377737 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:09:49 UTC) from {username='oryx', principal='oryx/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:58194
I20260807 05:09:49.852818 16579 raft_consensus.cc:2229] T bd713536d4a44f9793a993d7ee377737 P 52f3e3469f5d4c1cbe62d8084af37273 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:49.852844 16581 tablet_replica.cc:330] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250: stopping tablet replica
I20260807 05:09:49.852927 16579 raft_consensus.cc:2258] T bd713536d4a44f9793a993d7ee377737 P 52f3e3469f5d4c1cbe62d8084af37273 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:49.852952 16581 raft_consensus.cc:2229] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:09:49.853071 16580 tablet_replica.cc:330] T bd713536d4a44f9793a993d7ee377737 P 196776ad987c4d29afccd36feb109f94: stopping tablet replica
I20260807 05:09:49.853104 16581 raft_consensus.cc:2258] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:49.853183 16580 raft_consensus.cc:2229] T bd713536d4a44f9793a993d7ee377737 P 196776ad987c4d29afccd36feb109f94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:09:49.853284 16580 raft_consensus.cc:2258] T bd713536d4a44f9793a993d7ee377737 P 196776ad987c4d29afccd36feb109f94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:09:49.853719 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 16056
I20260807 05:09:49.853833 16581 ts_tablet_manager.cc:1905] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:49.856000 16580 ts_tablet_manager.cc:1905] T bd713536d4a44f9793a993d7ee377737 P 196776ad987c4d29afccd36feb109f94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:09:49.856266 16581 ts_tablet_manager.cc:1918] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:09:49.856331 16581 log.cc:1196] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/bd713536d4a44f9793a993d7ee377737
I20260807 05:09:49.856551 16581 ts_tablet_manager.cc:1939] T bd713536d4a44f9793a993d7ee377737 P 2fb9a84dbaa84054b1607e6759c47250: Deleting consensus metadata
I20260807 05:09:49.857127 15966 catalog_manager.cc:4841] TS 2fb9a84dbaa84054b1607e6759c47250 (127.8.16.67:42769): tablet bd713536d4a44f9793a993d7ee377737 (table test-table [id=f34f32a8656a4030884690eaf2f20f2d]) successfully deleted
I20260807 05:09:49.858341 16580 ts_tablet_manager.cc:1918] T bd713536d4a44f9793a993d7ee377737 P 196776ad987c4d29afccd36feb109f94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:09:49.858412 16580 log.cc:1196] T bd713536d4a44f9793a993d7ee377737 P 196776ad987c4d29afccd36feb109f94: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079354071073-8257-0/minicluster-data/ts-1/wal/wals/bd713536d4a44f9793a993d7ee377737
I20260807 05:09:49.858613 16580 ts_tablet_manager.cc:1939] T bd713536d4a44f9793a993d7ee377737 P 196776ad987c4d29afccd36feb109f94: Deleting consensus metadata
I20260807 05:09:49.859163 15963 catalog_manager.cc:4841] TS 196776ad987c4d29afccd36feb109f94 (127.8.16.66:33539): tablet bd713536d4a44f9793a993d7ee377737 (table test-table [id=f34f32a8656a4030884690eaf2f20f2d]) successfully deleted
W20260807 05:09:49.863193 15965 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.16.65:39621 (error 108)
I20260807 05:09:49.864003 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 16199
I20260807 05:09:49.864182 16611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:49.863827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:39621 (local address 127.0.0.1:49744)
0807 05:09:49.864024 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:49.864092 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.65:39621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
W20260807 05:09:49.865118 15965 catalog_manager.cc:4584] TS 52f3e3469f5d4c1cbe62d8084af37273 (127.8.16.65:39621): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bd713536d4a44f9793a993d7ee377737: Network error: Client connection negotiation failed: client connection to 127.8.16.65:39621: connect: Connection refused (error 111)
I20260807 05:09:49.869969 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 16352
I20260807 05:09:49.876773 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 15945
2026-08-07T05:09:49Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3079 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:09:49 dist-test-slave-49h3 krb5kdc[16614](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:09:49 dist-test-slave-49h3 krb5kdc[16614](info): set up 2 sockets
Aug 07 05:09:49 dist-test-slave-49h3 krb5kdc[16614](info): commencing operation
krb5kdc: starting...
W20260807 05:09:50.861807 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.963s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:09:50 dist-test-slave-49h3 krb5kdc[16614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:09:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:09:50.978456 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:39675
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:40315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:39675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:51.086174 16628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:51.086448 16628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:51.086488 16628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:51.090405 16628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:51.090459 16628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:51.090477 16628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:51.090503 16628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:51.090531 16628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:51.095973 16628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:40315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:39675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:39675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:51.097028 16628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:51.098049 16628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:51.106153 16633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:51.106148 16636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:51.106163 16634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:51.106364 16628 server_base.cc:1034] running on GCE node
I20260807 05:09:51.106974 16628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:51.108021 16628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:51.109194 16628 hybrid_clock.cc:648] HybridClock initialized: now 1786079391109176 us; error 29 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T05:09:51Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1239 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:09:51 dist-test-slave-49h3 krb5kdc[16643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:09:51 dist-test-slave-49h3 krb5kdc[16643](info): set up 2 sockets
Aug 07 05:09:51 dist-test-slave-49h3 krb5kdc[16643](info): commencing operation
krb5kdc: starting...
W20260807 05:09:52.089880 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.951s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:09:52 dist-test-slave-49h3 krb5kdc[16643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:09:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:09:52.203824 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:42575
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:43169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:42575
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:52.311257 16657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:52.311510 16657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:52.311565 16657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:52.315483 16657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:52.315539 16657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:52.315567 16657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:52.315586 16657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:52.315620 16657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:52.320719 16657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:43169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:42575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:42575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:52.321723 16657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:52.322680 16657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:52.330816 16663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:52.330819 16662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:52.330819 16665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:52.331007 16657 server_base.cc:1034] running on GCE node
I20260807 05:09:52.331712 16657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:52.332791 16657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:52.333946 16657 hybrid_clock.cc:648] HybridClock initialized: now 1786079392333923 us; error 38 us; skew 500 ppm
Aug 07 05:09:52 dist-test-slave-49h3 krb5kdc[16643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079392, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:52.341017 16657 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:09:52.342057 16657 webserver.cc:460] Webserver started at http://127.8.16.126:46429/ using document root <none> and password file <none>
I20260807 05:09:52.342605 16657 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:52.342675 16657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:52.342887 16657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:52.344919 16657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "f927117e985e45d18020841abd318020"
format_stamp: "Formatted at 2026-08-07 05:09:52 on dist-test-slave-49h3"
server_key: "177af148684bc94482e6c38c029d78b3"
server_key_iv: "11b4c53c27636aa818eb05726559133f"
server_key_version: "encryptionkey@0"
I20260807 05:09:52.345482 16657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "f927117e985e45d18020841abd318020"
format_stamp: "Formatted at 2026-08-07 05:09:52 on dist-test-slave-49h3"
server_key: "177af148684bc94482e6c38c029d78b3"
server_key_iv: "11b4c53c27636aa818eb05726559133f"
server_key_version: "encryptionkey@0"
I20260807 05:09:52.349031 16657 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:09:52.351380 16672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:52.352515 16657 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:52.352624 16657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "f927117e985e45d18020841abd318020"
format_stamp: "Formatted at 2026-08-07 05:09:52 on dist-test-slave-49h3"
server_key: "177af148684bc94482e6c38c029d78b3"
server_key_iv: "11b4c53c27636aa818eb05726559133f"
server_key_version: "encryptionkey@0"
I20260807 05:09:52.352712 16657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:52.370009 16657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:52.372668 16657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:52.372849 16657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:52.379565 16657 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:42575
I20260807 05:09:52.379581 16726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:42575 every 8 connection(s)
I20260807 05:09:52.380461 16657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:52.383059 16727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:52.388260 16727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020: Bootstrap starting.
I20260807 05:09:52.389932 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 16657
I20260807 05:09:52.390055 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:09:52.390309 8257 external_mini_cluster.cc:1428] Setting key 3d50db624261e36ea8cce9a628b75299
I20260807 05:09:52.390617 16727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:52.391445 16727 log.cc:824] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:52.393395 16727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020: No bootstrap required, opened a new log
I20260807 05:09:52.397410 16727 raft_consensus.cc:348] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f927117e985e45d18020841abd318020" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42575 } }
I20260807 05:09:52.397723 16727 raft_consensus.cc:374] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:52.397785 16727 raft_consensus.cc:729] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f927117e985e45d18020841abd318020, State: Initialized, Role: FOLLOWER
I20260807 05:09:52.398308 16727 consensus_queue.cc:260] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f927117e985e45d18020841abd318020" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42575 } }
I20260807 05:09:52.398473 16727 raft_consensus.cc:388] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:09:52.398546 16727 raft_consensus.cc:482] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:09:52.398631 16727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [term 0 FOLLOWER]: Advancing to term 1
Aug 07 05:09:52 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:52.399664 16727 raft_consensus.cc:504] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f927117e985e45d18020841abd318020" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42575 } }
I20260807 05:09:52.399999 16727 leader_election.cc:302] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f927117e985e45d18020841abd318020; no voters:
I20260807 05:09:52.400352 16727 leader_election.cc:288] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:09:52.400449 16732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:52.400615 16732 raft_consensus.cc:686] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [term 1 LEADER]: Becoming Leader. State: Replica: f927117e985e45d18020841abd318020, State: Running, Role: LEADER
I20260807 05:09:52.400906 16732 consensus_queue.cc:237] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f927117e985e45d18020841abd318020" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42575 } }
I20260807 05:09:52.401674 16727 sys_catalog.cc:564] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:52.402364 16734 sys_catalog.cc:455] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f927117e985e45d18020841abd318020. Latest consensus state: current_term: 1 leader_uuid: "f927117e985e45d18020841abd318020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f927117e985e45d18020841abd318020" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42575 } } }
I20260807 05:09:52.402484 16734 sys_catalog.cc:458] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:52.402410 16733 sys_catalog.cc:455] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f927117e985e45d18020841abd318020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f927117e985e45d18020841abd318020" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42575 } } }
I20260807 05:09:52.402587 16733 sys_catalog.cc:458] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:52.404729 16730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:52.391354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37528 (local address 127.8.16.126:42575)
0807 05:09:52.391717 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:52.391740 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:52.391774 (+ 34us) server_negotiation.cc:407] Connection header received
0807 05:09:52.392485 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:52.392504 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:52.392819 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:52.393086 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:52.394026 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:52.395002 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:52.396190 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:52.396698 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:52.396825 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:52.399620 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:52.399625 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:52.399640 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:52.399669 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:52.401581 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:52.402061 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:52.402067 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:52.402073 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:52.402203 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:52.402549 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:52.402552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:52.402554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:52.402996 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:52.403192 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:52.403491 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 05:09:52.403787 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":94,"threads_started":1}
W20260807 05:09:52.407227 16748 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:09:52.407295 16748 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:09:52.407361 16742 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:52.408026 16742 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:52.413373 16742 catalog_manager.cc:1347] Generated new cluster ID: ccf2cca7d45d4cc2a331de9ffd2c2f91
I20260807 05:09:52.413475 16742 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:52.428153 16742 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:09:52.429417 16742 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:52.436561 16742 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020: Generated new TSK 0
I20260807 05:09:52.437330 16742 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
I20260807 05:09:52.469620 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:42575
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:43169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:52.578889 16754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:52.579186 16754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:52.579239 16754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:52.583151 16754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:52.583213 16754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:52.583276 16754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:09:52.588490 16754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:43169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:42575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:52.589504 16754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:52.590454 16754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:52.599437 16760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:52.599448 16762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:52.599444 16759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:52.599893 16754 server_base.cc:1034] running on GCE node
I20260807 05:09:52.600282 16754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:52.600914 16754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:52.602020 16754 hybrid_clock.cc:648] HybridClock initialized: now 1786079392601999 us; error 32 us; skew 500 ppm
Aug 07 05:09:52 dist-test-slave-49h3 krb5kdc[16643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079392, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:52.604655 16754 init.cc:377] Logged in from keytab as kudu/127.8.16.65@KRBTEST.COM (short username kudu)
I20260807 05:09:52.605546 16754 webserver.cc:460] Webserver started at http://127.8.16.65:33839/ using document root <none> and password file <none>
I20260807 05:09:52.606046 16754 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:52.606119 16754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:52.606298 16754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:52.608286 16754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "5daba5f02dd04579bcb96f12aae3973a"
format_stamp: "Formatted at 2026-08-07 05:09:52 on dist-test-slave-49h3"
server_key: "50b8d4baea781fa23b0381271f26793c"
server_key_iv: "5fff2806475ac087132ba6e5e2c662ef"
server_key_version: "encryptionkey@0"
I20260807 05:09:52.608778 16754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "5daba5f02dd04579bcb96f12aae3973a"
format_stamp: "Formatted at 2026-08-07 05:09:52 on dist-test-slave-49h3"
server_key: "50b8d4baea781fa23b0381271f26793c"
server_key_iv: "5fff2806475ac087132ba6e5e2c662ef"
server_key_version: "encryptionkey@0"
I20260807 05:09:52.612324 16754 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:09:52.614610 16770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:52.615777 16754 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:52.615882 16754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "5daba5f02dd04579bcb96f12aae3973a"
format_stamp: "Formatted at 2026-08-07 05:09:52 on dist-test-slave-49h3"
server_key: "50b8d4baea781fa23b0381271f26793c"
server_key_iv: "5fff2806475ac087132ba6e5e2c662ef"
server_key_version: "encryptionkey@0"
I20260807 05:09:52.615967 16754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:52.637431 16754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:52.640587 16754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:52.640753 16754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:52.641301 16754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:52.642119 16754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:52.642168 16754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:52.642205 16754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:52.642225 16754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:52.650298 16754 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:35423
I20260807 05:09:52.650333 16890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:35423 every 8 connection(s)
I20260807 05:09:52.651154 16754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:09:52.656100 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 16754
I20260807 05:09:52.656239 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:09:52.656497 8257 external_mini_cluster.cc:1428] Setting key 7a92fe90c05235881129ab0d350c5316
Aug 07 05:09:52 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079392, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:52.665071 16730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:52.652812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:42797 (local address 127.8.16.126:42575)
0807 05:09:52.652944 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:52.652948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:52.653801 (+ 853us) server_negotiation.cc:407] Connection header received
0807 05:09:52.655075 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:52.655079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:52.655139 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:52.655211 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:52.656495 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:52.657183 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:52.658667 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:52.658899 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:52.658974 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:52.661572 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:52.661576 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:52.661579 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:52.661607 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:52.663093 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:52.663584 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:52.663587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:52.663588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:52.663667 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:52.663983 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:52.663987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:52.663988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:52.664165 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:52.664250 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:52.664754 (+ 504us) server_negotiation.cc:299] Negotiation successful
0807 05:09:52.664896 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:09:52.665858 16893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:52.653153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.65:42797)
0807 05:09:52.653641 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:52.653686 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:52.654862 (+ 1176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:52.655389 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:52.655396 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:52.655774 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:52.656353 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:52.656370 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:52.657315 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:52.657318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:52.658538 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:52.658549 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:52.659035 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:52.659039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:52.659220 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:52.659829 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:52.659850 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:52.661393 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:52.663201 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:52.663206 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:52.663217 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:52.663473 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:52.663749 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:52.663752 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:52.663755 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:52.663886 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:52.664271 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:52.664278 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:52.664511 (+ 233us) client_negotiation.cc:769] Sending connection context
0807 05:09:52.664668 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 05:09:52.664991 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":130,"threads_started":1}
WARNING: no policy specified for kudu/127.8.16.66@KRBTEST.COM; defaulting to no policy
I20260807 05:09:52.667038 16891 heartbeater.cc:347] Connected to a master server at 127.8.16.126:42575
I20260807 05:09:52.667284 16891 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:52.667780 16891 heartbeater.cc:511] Master 127.8.16.126:42575 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.66@KRBTEST.COM" created.
I20260807 05:09:52.669414 16690 ts_manager.cc:194] Registered new tserver with Master: 5daba5f02dd04579bcb96f12aae3973a (127.8.16.65:35423)
I20260807 05:09:52.670893 16690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:42797
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
I20260807 05:09:52.684163 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:42575
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:43169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:52.814568 16896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:52.814836 16896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:52.814893 16896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:52.818878 16896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:52.818941 16896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:52.819016 16896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:09:52.824224 16896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:43169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:42575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:52.825271 16896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:52.826341 16896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:52.835405 16904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:52.835402 16902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:52.835436 16901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:52.835728 16896 server_base.cc:1034] running on GCE node
I20260807 05:09:52.836120 16896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:52.836753 16896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:52.837942 16896 hybrid_clock.cc:648] HybridClock initialized: now 1786079392837921 us; error 30 us; skew 500 ppm
Aug 07 05:09:52 dist-test-slave-49h3 krb5kdc[16643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079392, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:52.840528 16896 init.cc:377] Logged in from keytab as kudu/127.8.16.66@KRBTEST.COM (short username kudu)
I20260807 05:09:52.841367 16896 webserver.cc:460] Webserver started at http://127.8.16.66:33789/ using document root <none> and password file <none>
I20260807 05:09:52.841914 16896 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:52.841981 16896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:52.842168 16896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:52.844167 16896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "3f3b57fe3ee84d5caa1aaf5d3f8b6830"
format_stamp: "Formatted at 2026-08-07 05:09:52 on dist-test-slave-49h3"
server_key: "c23d8a07e38372dbb352c66b85959ef1"
server_key_iv: "fc59ff321bc5f0d7c04ef3486f867cbf"
server_key_version: "encryptionkey@0"
I20260807 05:09:52.844704 16896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "3f3b57fe3ee84d5caa1aaf5d3f8b6830"
format_stamp: "Formatted at 2026-08-07 05:09:52 on dist-test-slave-49h3"
server_key: "c23d8a07e38372dbb352c66b85959ef1"
server_key_iv: "fc59ff321bc5f0d7c04ef3486f867cbf"
server_key_version: "encryptionkey@0"
I20260807 05:09:52.848227 16896 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260807 05:09:52.850485 16913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:52.851722 16896 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:52.851835 16896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "3f3b57fe3ee84d5caa1aaf5d3f8b6830"
format_stamp: "Formatted at 2026-08-07 05:09:52 on dist-test-slave-49h3"
server_key: "c23d8a07e38372dbb352c66b85959ef1"
server_key_iv: "fc59ff321bc5f0d7c04ef3486f867cbf"
server_key_version: "encryptionkey@0"
I20260807 05:09:52.851929 16896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:52.868490 16896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:52.871322 16896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:52.871498 16896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:52.872068 16896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:52.872866 16896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:52.872915 16896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:52.872967 16896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:52.872994 16896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:52.881327 16896 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:43149
I20260807 05:09:52.881350 17032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:43149 every 8 connection(s)
I20260807 05:09:52.882221 16896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
Aug 07 05:09:52 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079392, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:52.891841 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 16896
I20260807 05:09:52.891961 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:09:52.892307 8257 external_mini_cluster.cc:1428] Setting key e817a02dc9a958f19978ec41afbfb4db
I20260807 05:09:52.895296 16730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:52.883922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:34095 (local address 127.8.16.126:42575)
0807 05:09:52.884054 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:52.884057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:52.884832 (+ 775us) server_negotiation.cc:407] Connection header received
0807 05:09:52.886132 (+ 1300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:52.886135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:52.886197 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:52.886277 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:52.887412 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:52.887939 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:52.889036 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:52.889246 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:52.889334 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:52.891663 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:52.891666 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:52.891668 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:52.891692 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:52.893162 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:52.893801 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:52.893804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:52.893805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:52.893894 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:52.894236 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:52.894239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:52.894241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:52.894469 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:52.894560 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:52.895017 (+ 457us) server_negotiation.cc:299] Negotiation successful
0807 05:09:52.895143 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 05:09:52.896111 17035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:52.884262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.66:34095)
0807 05:09:52.884690 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:52.884730 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:52.885911 (+ 1181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:52.886449 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:52.886456 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:52.886799 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:52.887272 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:52.887285 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:52.888057 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:52.888060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:52.888919 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:52.888928 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:52.889382 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:52.889387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:52.889581 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:52.890111 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:52.890127 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:52.891492 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:52.893278 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:52.893283 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:52.893296 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:52.893697 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:52.893995 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:52.893998 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:52.893999 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:52.894133 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:52.894560 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:52.894565 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:52.894792 (+ 227us) client_negotiation.cc:769] Sending connection context
0807 05:09:52.894932 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 05:09:52.895178 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":116,"threads_started":1}
I20260807 05:09:52.897199 17033 heartbeater.cc:347] Connected to a master server at 127.8.16.126:42575
I20260807 05:09:52.897426 17033 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:52.897917 17033 heartbeater.cc:511] Master 127.8.16.126:42575 requested a full tablet report, sending...
I20260807 05:09:52.898900 16690 ts_manager.cc:194] Registered new tserver with Master: 3f3b57fe3ee84d5caa1aaf5d3f8b6830 (127.8.16.66:43149)
I20260807 05:09:52.899434 16690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:34095
WARNING: no policy specified for kudu/127.8.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
I20260807 05:09:52.918063 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:42575
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:43169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:53.026190 17038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:53.026437 17038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:53.026480 17038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:53.030366 17038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:53.030418 17038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:53.030485 17038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:09:53.035753 17038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:43169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:42575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:53.036705 17038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:53.037691 17038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:53.046816 17043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:53.046820 17046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:53.046913 17038 server_base.cc:1034] running on GCE node
W20260807 05:09:53.046816 17044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:53.047312 17038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:53.047935 17038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:53.049082 17038 hybrid_clock.cc:648] HybridClock initialized: now 1786079393049057 us; error 38 us; skew 500 ppm
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079393, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:53.051622 17038 init.cc:377] Logged in from keytab as kudu/127.8.16.67@KRBTEST.COM (short username kudu)
I20260807 05:09:53.052453 17038 webserver.cc:460] Webserver started at http://127.8.16.67:35807/ using document root <none> and password file <none>
I20260807 05:09:53.052973 17038 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:53.053037 17038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:53.053231 17038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:53.055250 17038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "25aedb5254a64f0ab0eba9122939ece7"
format_stamp: "Formatted at 2026-08-07 05:09:53 on dist-test-slave-49h3"
server_key: "f6a2906e464c0c4b3562c3592de138d1"
server_key_iv: "35a7284e84e1424ac4e7d56084756297"
server_key_version: "encryptionkey@0"
I20260807 05:09:53.055863 17038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "25aedb5254a64f0ab0eba9122939ece7"
format_stamp: "Formatted at 2026-08-07 05:09:53 on dist-test-slave-49h3"
server_key: "f6a2906e464c0c4b3562c3592de138d1"
server_key_iv: "35a7284e84e1424ac4e7d56084756297"
server_key_version: "encryptionkey@0"
I20260807 05:09:53.059478 17038 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:09:53.061714 17053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:53.062819 17038 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:09:53.062923 17038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "25aedb5254a64f0ab0eba9122939ece7"
format_stamp: "Formatted at 2026-08-07 05:09:53 on dist-test-slave-49h3"
server_key: "f6a2906e464c0c4b3562c3592de138d1"
server_key_iv: "35a7284e84e1424ac4e7d56084756297"
server_key_version: "encryptionkey@0"
I20260807 05:09:53.063012 17038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:53.084218 17038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:53.086988 17038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:53.087167 17038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:53.087745 17038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:53.088532 17038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:53.088580 17038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:53.088631 17038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:53.088665 17038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:53.096777 17038 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:44383
I20260807 05:09:53.096813 17172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:44383 every 8 connection(s)
I20260807 05:09:53.097689 17038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:09:53.104482 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 17038
I20260807 05:09:53.104622 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:09:53.104933 8257 external_mini_cluster.cc:1428] Setting key dc88ba446c6626611f48e97307cb12fb
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079393, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:09:53.110360 16730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:53.099358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:40689 (local address 127.8.16.126:42575)
0807 05:09:53.099486 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:53.099490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:53.100297 (+ 807us) server_negotiation.cc:407] Connection header received
0807 05:09:53.101555 (+ 1258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:53.101557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:53.101606 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:53.101676 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:53.102831 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.103392 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.104493 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.104819 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.104921 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:53.107386 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:53.107389 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:53.107391 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:53.107416 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:53.108749 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:53.109142 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:53.109145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:53.109146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:53.109215 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:53.109495 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:53.109498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:53.109499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:53.109664 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:53.109745 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:53.110143 (+ 398us) server_negotiation.cc:299] Negotiation successful
0807 05:09:53.110238 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 05:09:53.111171 17176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:53.099718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.67:40689)
0807 05:09:53.100137 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:53.100195 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:53.101351 (+ 1156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:53.101814 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:53.101820 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:53.102208 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:53.102706 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:53.102720 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.103486 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.103489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:53.104383 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:53.104392 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.104952 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.104955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:53.105113 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:53.105717 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:53.105740 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:53.107245 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:53.108828 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:53.108832 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:53.108843 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:53.109075 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:53.109276 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:53.109278 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:53.109279 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:53.109407 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:53.109735 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:53.109740 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:53.109960 (+ 220us) client_negotiation.cc:769] Sending connection context
0807 05:09:53.110102 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 05:09:53.110357 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":117,"threads_started":1}
I20260807 05:09:53.112241 17174 heartbeater.cc:347] Connected to a master server at 127.8.16.126:42575
I20260807 05:09:53.112452 17174 heartbeater.cc:464] Registering TS with master...
I20260807 05:09:53.112906 17174 heartbeater.cc:511] Master 127.8.16.126:42575 requested a full tablet report, sending...
I20260807 05:09:53.113850 16690 ts_manager.cc:194] Registered new tserver with Master: 25aedb5254a64f0ab0eba9122939ece7 (127.8.16.67:44383)
I20260807 05:09:53.114449 16690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:40689
I20260807 05:09:53.118592 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.65.keytab.
Entry for principal oryx/127.8.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.65.keytab.
WARNING: no policy specified for oryx/127.8.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.66.keytab.
Entry for principal oryx/127.8.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.66.keytab.
WARNING: no policy specified for oryx/127.8.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.67.keytab.
Entry for principal oryx/127.8.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.67.keytab.
I20260807 05:09:53.183518 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 16754
I20260807 05:09:53.189651 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 16896
I20260807 05:09:53.197499 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 17038
I20260807 05:09:53.203238 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 16657
I20260807 05:09:53.208930 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:42575
--webserver_interface=127.8.16.126
--webserver_port=46429
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:43169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:42575
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:53.316422 17183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:53.316687 17183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:53.316751 17183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:53.320719 17183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:53.320786 17183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:53.320813 17183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:53.320834 17183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:53.320859 17183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:53.326105 17183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:43169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:42575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:42575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=46429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:53.327113 17183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:53.328083 17183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:53.335879 17189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:53.335891 17188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:53.335881 17192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:53.336040 17183 server_base.cc:1034] running on GCE node
I20260807 05:09:53.336699 17183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:53.337752 17183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:53.338907 17183 hybrid_clock.cc:648] HybridClock initialized: now 1786079393338879 us; error 41 us; skew 500 ppm
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079393, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:53.341899 17183 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:09:53.343019 17183 webserver.cc:460] Webserver started at http://127.8.16.126:46429/ using document root <none> and password file <none>
I20260807 05:09:53.343585 17183 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:53.343664 17183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:53.347498 17183 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 05:09:53.349547 17199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:53.350728 17183 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:53.350850 17183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "f927117e985e45d18020841abd318020"
format_stamp: "Formatted at 2026-08-07 05:09:52 on dist-test-slave-49h3"
server_key: "177af148684bc94482e6c38c029d78b3"
server_key_iv: "11b4c53c27636aa818eb05726559133f"
server_key_version: "encryptionkey@0"
I20260807 05:09:53.351367 17183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:53.375573 17183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:53.378330 17183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:53.378508 17183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:53.385159 17183 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:42575
I20260807 05:09:53.385177 17255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:42575 every 8 connection(s)
I20260807 05:09:53.386281 17183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:53.390342 17256 sys_catalog.cc:263] Verifying existing consensus state
I20260807 05:09:53.391911 17256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020: Bootstrap starting.
I20260807 05:09:53.394944 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 17183
I20260807 05:09:53.395345 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:35423
--local_ip_for_outbound_sockets=127.8.16.65
--tserver_master_addrs=127.8.16.126:42575
--webserver_port=33839
--webserver_interface=127.8.16.65
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:43169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.16.65
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:09:53.407981 17256 log.cc:824] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:53.414345 17256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:09:53.414680 17256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020: Bootstrap complete.
I20260807 05:09:53.418604 17256 raft_consensus.cc:348] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f927117e985e45d18020841abd318020" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42575 } }
I20260807 05:09:53.419019 17256 raft_consensus.cc:729] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f927117e985e45d18020841abd318020, State: Initialized, Role: FOLLOWER
I20260807 05:09:53.419500 17256 consensus_queue.cc:260] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f927117e985e45d18020841abd318020" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42575 } }
I20260807 05:09:53.419615 17256 raft_consensus.cc:388] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:09:53.419678 17256 raft_consensus.cc:482] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:09:53.419778 17256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:09:53.421705 17256 raft_consensus.cc:504] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f927117e985e45d18020841abd318020" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42575 } }
I20260807 05:09:53.422001 17256 leader_election.cc:302] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f927117e985e45d18020841abd318020; no voters:
I20260807 05:09:53.422282 17256 leader_election.cc:288] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 05:09:53.422394 17260 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 05:09:53.422567 17260 raft_consensus.cc:686] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [term 2 LEADER]: Becoming Leader. State: Replica: f927117e985e45d18020841abd318020, State: Running, Role: LEADER
I20260807 05:09:53.422847 17260 consensus_queue.cc:237] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f927117e985e45d18020841abd318020" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42575 } }
I20260807 05:09:53.423488 17256 sys_catalog.cc:564] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:53.423758 17260 sys_catalog.cc:455] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f927117e985e45d18020841abd318020. Latest consensus state: current_term: 2 leader_uuid: "f927117e985e45d18020841abd318020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f927117e985e45d18020841abd318020" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42575 } } }
I20260807 05:09:53.423954 17260 sys_catalog.cc:458] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:53.425065 17264 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:53.425206 17261 sys_catalog.cc:455] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f927117e985e45d18020841abd318020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f927117e985e45d18020841abd318020" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42575 } } }
I20260807 05:09:53.425364 17261 sys_catalog.cc:458] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:53.428035 17264 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:53.428648 17264 catalog_manager.cc:1259] Loaded cluster ID: ccf2cca7d45d4cc2a331de9ffd2c2f91
I20260807 05:09:53.428711 17264 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:53.430998 17264 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:53.431645 17264 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f927117e985e45d18020841abd318020: Loaded TSK: 0
I20260807 05:09:53.432807 17264 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 05:09:53.505923 17258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:53.506202 17258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:53.506264 17258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:53.510196 17258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:53.510253 17258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:53.510327 17258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:09:53.515444 17258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:43169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.65.keytab
--principal=oryx/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:35423
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=33839
--tserver_master_addrs=127.8.16.126:42575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:53.516402 17258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:53.517352 17258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:53.527527 17287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:53.527537 17285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:53.527537 17284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:53.527719 17258 server_base.cc:1034] running on GCE node
I20260807 05:09:53.528290 17258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:53.528903 17258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:53.530064 17258 hybrid_clock.cc:648] HybridClock initialized: now 1786079393530051 us; error 27 us; skew 500 ppm
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079393, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:53.532879 17258 init.cc:377] Logged in from keytab as oryx/127.8.16.65@KRBTEST.COM (short username oryx)
I20260807 05:09:53.533769 17258 webserver.cc:460] Webserver started at http://127.8.16.65:33839/ using document root <none> and password file <none>
I20260807 05:09:53.534296 17258 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:53.534368 17258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:53.538188 17258 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:09:53.540164 17294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:53.541388 17258 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:53.541507 17258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "5daba5f02dd04579bcb96f12aae3973a"
format_stamp: "Formatted at 2026-08-07 05:09:52 on dist-test-slave-49h3"
server_key: "50b8d4baea781fa23b0381271f26793c"
server_key_iv: "5fff2806475ac087132ba6e5e2c662ef"
server_key_version: "encryptionkey@0"
I20260807 05:09:53.542022 17258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:53.558763 17258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:53.561787 17258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:53.561955 17258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:53.562505 17258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:53.563308 17258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:53.563354 17258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:53.563396 17258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:53.563426 17258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:53.571513 17258 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:35423
I20260807 05:09:53.571547 17413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:35423 every 8 connection(s)
I20260807 05:09:53.572460 17258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:09:53.582052 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 17258
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:53.582499 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:43149
--local_ip_for_outbound_sockets=127.8.16.66
--tserver_master_addrs=127.8.16.126:42575
--webserver_port=33789
--webserver_interface=127.8.16.66
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:43169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.16.66
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 05:09:53.584713 17416 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:53.574395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:51373 (local address 127.8.16.126:42575)
0807 05:09:53.574861 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:53.574881 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:53.575090 (+ 209us) server_negotiation.cc:407] Connection header received
0807 05:09:53.576493 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:53.576503 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:53.576772 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:53.577054 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:53.578174 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.579116 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.580202 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.580656 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.580801 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:53.584032 (+ 3231us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.65:51373: BlockingRecv error: recv got EOF from 127.8.16.65:51373 (error 108)
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":131,"threads_started":1}
I20260807 05:09:53.584720 17418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:53.574496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.65:51373)
0807 05:09:53.574947 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:53.574987 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:53.576121 (+ 1134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:53.577146 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:53.577154 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:53.577573 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:53.578013 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:53.578026 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.579245 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.579248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:53.580083 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:53.580091 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.580813 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.580817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:53.580983 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:53.581496 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:53.581511 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:53.583797 (+ 2286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":134,"threads_started":1}
W20260807 05:09:53.585386 17414 heartbeater.cc:644] Failed to heartbeat to 127.8.16.126:42575 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.16.126:42575: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:53.590507 17418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:53.585820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.65:59793)
0807 05:09:53.585922 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:53.585937 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:53.586005 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:53.586314 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:53.586316 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:53.586458 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:53.586540 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:53.586545 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.587218 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.587222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:53.587792 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:53.587800 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.588198 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.588201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:53.588286 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:53.588751 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:53.588761 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:53.590386 (+ 1625us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 05:09:53.590613 17416 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:53.585895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:59793 (local address 127.8.16.126:42575)
0807 05:09:53.586132 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:53.586135 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:53.586144 (+ 9us) server_negotiation.cc:407] Connection header received
0807 05:09:53.586173 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:53.586175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:53.586209 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:53.586280 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:53.586642 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.587124 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.587900 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.588106 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.588189 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:53.590515 (+ 2326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.65:59793: BlockingRecv error: recv got EOF from 127.8.16.65:59793 (error 108)
Metrics: {"server-negotiator.queue_time_us":159}
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:53.595443 17418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:53.591154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.65:48149)
0807 05:09:53.591269 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:53.591282 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:53.591361 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:53.591558 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:53.591561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:53.591692 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:53.591758 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:53.591762 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.592409 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.592412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:53.592901 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:53.592908 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.593297 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.593300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:53.593372 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:53.593806 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:53.593816 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:53.595326 (+ 1510us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 05:09:53.595527 17416 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:53.591199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:48149 (local address 127.8.16.126:42575)
0807 05:09:53.591293 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:53.591297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:53.591325 (+ 28us) server_negotiation.cc:407] Connection header received
0807 05:09:53.591432 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:53.591435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:53.591474 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:53.591536 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:53.591853 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.592317 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.592999 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.593207 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.593288 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:53.595430 (+ 2142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.65:48149: BlockingRecv error: recv got EOF from 127.8.16.65:48149 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260807 05:09:53.595672 17414 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 05:09:53.692322 17419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:53.692593 17419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:53.692636 17419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:53.696651 17419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:53.696734 17419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:53.696794 17419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:09:53.702549 17419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:43169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.66.keytab
--principal=oryx/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:43149
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=33789
--tserver_master_addrs=127.8.16.126:42575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:53.703598 17419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:53.704550 17419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:53.713671 17424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:53.713687 17427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:53.713687 17425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:53.713855 17419 server_base.cc:1034] running on GCE node
I20260807 05:09:53.714315 17419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:53.714881 17419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:53.716035 17419 hybrid_clock.cc:648] HybridClock initialized: now 1786079393716016 us; error 32 us; skew 500 ppm
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079393, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:53.718650 17419 init.cc:377] Logged in from keytab as oryx/127.8.16.66@KRBTEST.COM (short username oryx)
I20260807 05:09:53.719455 17419 webserver.cc:460] Webserver started at http://127.8.16.66:33789/ using document root <none> and password file <none>
I20260807 05:09:53.719969 17419 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:53.720048 17419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:53.723927 17419 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:09:53.726033 17434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:53.727234 17419 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:53.727341 17419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "3f3b57fe3ee84d5caa1aaf5d3f8b6830"
format_stamp: "Formatted at 2026-08-07 05:09:52 on dist-test-slave-49h3"
server_key: "c23d8a07e38372dbb352c66b85959ef1"
server_key_iv: "fc59ff321bc5f0d7c04ef3486f867cbf"
server_key_version: "encryptionkey@0"
I20260807 05:09:53.727865 17419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:53.748517 17419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:53.751724 17419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:53.751911 17419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:53.752522 17419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:53.753355 17419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:53.753405 17419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:53.753484 17419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:53.753515 17419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:53.762171 17419 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:43149
I20260807 05:09:53.762193 17554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:43149 every 8 connection(s)
I20260807 05:09:53.763166 17419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:09:53.769640 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 17419
I20260807 05:09:53.770104 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:44383
--local_ip_for_outbound_sockets=127.8.16.67
--tserver_master_addrs=127.8.16.126:42575
--webserver_port=35807
--webserver_interface=127.8.16.67
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:43169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.16.67
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 05:09:53.773773 17416 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:53.764897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:37563 (local address 127.8.16.126:42575)
0807 05:09:53.765081 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:53.765086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:53.765910 (+ 824us) server_negotiation.cc:407] Connection header received
0807 05:09:53.767118 (+ 1208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:53.767120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:53.767170 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:53.767241 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:53.768321 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.768846 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.770024 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.770332 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.770433 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:53.773648 (+ 3215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.66:37563: BlockingRecv error: recv got EOF from 127.8.16.66:37563 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 05:09:53.774358 17557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:53.765296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.66:37563)
0807 05:09:53.765757 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:53.765798 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:53.766937 (+ 1139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:53.767370 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:53.767376 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:53.767727 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:53.768204 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:53.768217 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.768935 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.768937 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:53.769908 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:53.769918 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.770470 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.770473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:53.770617 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:53.771164 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:53.771181 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:53.773498 (+ 2317us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":142,"threads_started":1}
W20260807 05:09:53.774811 17555 heartbeater.cc:644] Failed to heartbeat to 127.8.16.126:42575 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.16.126:42575: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:53.779903 17557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:53.775244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.66:49885)
0807 05:09:53.775381 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:53.775392 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:53.775464 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:53.775674 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:53.775677 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:53.775812 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:53.775898 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:53.775903 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.776587 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.776589 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:53.777116 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:53.777123 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.777559 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.777562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:53.777656 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:53.778074 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:53.778085 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:53.779787 (+ 1702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":85}
W20260807 05:09:53.780015 17416 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:53.775354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:49885 (local address 127.8.16.126:42575)
0807 05:09:53.775473 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:53.775476 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:53.775489 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:09:53.775539 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:53.775542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:53.775579 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:53.775639 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:53.775999 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.776493 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.777219 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.777436 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.777552 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:53.779915 (+ 2363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.66:49885: BlockingRecv error: recv got EOF from 127.8.16.66:49885 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:53.785434 17557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:53.780558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.66:47817)
0807 05:09:53.780657 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:53.780671 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:53.780737 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:53.780958 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:53.780961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:53.781104 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:53.781212 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:53.781217 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.782197 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.782199 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:53.782691 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:53.782698 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.783198 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.783201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:53.783251 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:53.783659 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:53.783673 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:53.785342 (+ 1669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 05:09:53.785632 17555 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 05:09:53.785825 17416 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:53.780646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:47817 (local address 127.8.16.126:42575)
0807 05:09:53.780757 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:53.780761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:53.780774 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:09:53.780808 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:53.780811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:53.780849 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:53.780921 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:53.781310 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.782100 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.782779 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.783109 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.783193 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:53.785517 (+ 2324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.66:47817: BlockingRecv error: recv got EOF from 127.8.16.66:47817 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 05:09:53.882088 17558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:53.882354 17558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:53.882397 17558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:53.886334 17558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:53.886386 17558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:53.886443 17558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:09:53.891554 17558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:43169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/krb5kdc/oryx_127.8.16.67.keytab
--principal=oryx/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:44383
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=35807
--tserver_master_addrs=127.8.16.126:42575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:53.892513 17558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:53.893440 17558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:53.902618 17563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:53.902729 17558 server_base.cc:1034] running on GCE node
W20260807 05:09:53.902629 17564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:53.902637 17567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:53.903302 17558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:53.903873 17558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:53.905018 17558 hybrid_clock.cc:648] HybridClock initialized: now 1786079393905003 us; error 28 us; skew 500 ppm
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079393, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:53.907737 17558 init.cc:377] Logged in from keytab as oryx/127.8.16.67@KRBTEST.COM (short username oryx)
I20260807 05:09:53.908619 17558 webserver.cc:460] Webserver started at http://127.8.16.67:35807/ using document root <none> and password file <none>
I20260807 05:09:53.909169 17558 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:53.909250 17558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:53.912985 17558 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:53.914959 17575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:53.916105 17558 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:53.916224 17558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "25aedb5254a64f0ab0eba9122939ece7"
format_stamp: "Formatted at 2026-08-07 05:09:53 on dist-test-slave-49h3"
server_key: "f6a2906e464c0c4b3562c3592de138d1"
server_key_iv: "35a7284e84e1424ac4e7d56084756297"
server_key_version: "encryptionkey@0"
I20260807 05:09:53.916738 17558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:53.927031 17558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:53.929961 17558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:53.930125 17558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:53.930686 17558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:09:53.931473 17558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:09:53.931516 17558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:53.931558 17558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:09:53.931587 17558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:53.941686 17558 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:44383
I20260807 05:09:53.941745 17692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:44383 every 8 connection(s)
I20260807 05:09:53.942847 17558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:09:53.946980 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 17558
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 05:09:53.954829 17416 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:53.944795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:49827 (local address 127.8.16.126:42575)
0807 05:09:53.944928 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:53.944932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:53.945834 (+ 902us) server_negotiation.cc:407] Connection header received
0807 05:09:53.947293 (+ 1459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:53.947295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:53.947342 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:53.947431 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:53.948875 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.949510 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.950933 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.951252 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.951352 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:53.954704 (+ 3352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.67:49827: BlockingRecv error: recv got EOF from 127.8.16.67:49827 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 05:09:53.955377 17695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:53.945213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.67:49827)
0807 05:09:53.945685 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:53.945728 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:53.947074 (+ 1346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:53.947580 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:53.947589 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:53.948060 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:53.948737 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:53.948756 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.949616 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.949618 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:53.950812 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:53.950821 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.951379 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.951382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:53.951519 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:53.952051 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:53.952073 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:53.954565 (+ 2492us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":123,"threads_started":1}
W20260807 05:09:53.955955 17693 heartbeater.cc:644] Failed to heartbeat to 127.8.16.126:42575 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.16.126:42575: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
I20260807 05:09:53.957396 17697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:53.948011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37536 (local address 127.8.16.126:42575)
0807 05:09:53.949091 (+ 1080us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:53.949095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:53.949104 (+ 9us) server_negotiation.cc:407] Connection header received
0807 05:09:53.949133 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:53.949134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:53.949166 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:53.949237 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:53.950051 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.950814 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.951811 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.952084 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.952189 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:53.953393 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:53.953398 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:53.953412 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:53.953436 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:53.955216 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:53.955742 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:53.955748 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:53.955754 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:53.955897 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:53.956185 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:53.956189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:53.956191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:53.956605 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:53.956785 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:53.957015 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 05:09:53.957228 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1019,"thread_start_us":91,"threads_started":1}
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:53.961220 17695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:53.956338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.67:56935)
0807 05:09:53.956493 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:53.956505 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:53.956560 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:53.956754 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:53.956757 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:53.956908 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:53.957014 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:53.957021 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.957933 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.957936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:53.958623 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:53.958633 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.959033 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.959036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:53.959123 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:53.959534 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:53.959545 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:53.961130 (+ 1585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94}
W20260807 05:09:53.961334 17416 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:53.956428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:56935 (local address 127.8.16.126:42575)
0807 05:09:53.956535 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:53.956539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:53.956552 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:09:53.956617 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:53.956619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:53.956657 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:53.956721 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:53.957116 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.957838 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.958737 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.958935 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.959028 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:53.961242 (+ 2214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.67:56935: BlockingRecv error: recv got EOF from 127.8.16.67:56935 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:53 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:53.965663 17695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:53.961690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.67:37185)
0807 05:09:53.961792 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:53.961807 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:53.961876 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:53.962036 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:53.962038 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:53.962146 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:53.962211 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:53.962215 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.962835 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.962837 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:53.963340 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:53.963347 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.963672 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.963674 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:53.963725 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:53.964061 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:53.964071 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:53.965570 (+ 1499us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 05:09:53.965806 17416 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:53.961759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:37185 (local address 127.8.16.126:42575)
0807 05:09:53.961887 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:53.961890 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:53.961902 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:09:53.961929 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:53.961931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:53.961961 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:53.962019 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:53.962284 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.962742 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.963418 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:53.963601 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:53.963663 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:53.965703 (+ 2040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.67:37185: BlockingRecv error: recv got EOF from 127.8.16.67:37185 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 05:09:53.965911 17693 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 05:09:54 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:54 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:54.601572 17698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:54.596283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.65:33135)
0807 05:09:54.596558 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:54.596571 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:54.596677 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:54.596901 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:54.596903 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:54.597076 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:54.597174 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:54.597180 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:54.597882 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:54.597884 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:54.598459 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:54.598466 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:54.598830 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:54.598833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:54.598889 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:54.599421 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:54.599433 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:54.601404 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":88,"threads_started":1}
W20260807 05:09:54.601697 17699 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:54.596364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:33135 (local address 127.8.16.126:42575)
0807 05:09:54.596581 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:54.596584 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:54.596624 (+ 40us) server_negotiation.cc:407] Connection header received
0807 05:09:54.596764 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:54.596766 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:54.596810 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:54.596876 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:54.597256 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:54.597805 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:54.598539 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:54.598756 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:54.598821 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:54.601569 (+ 2748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.65:33135: BlockingRecv error: recv got EOF from 127.8.16.65:33135 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":65,"threads_started":1}
Aug 07 05:09:54 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:54 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:54.791226 17700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:54.786215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.66:52105)
0807 05:09:54.786492 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:54.786504 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:54.786610 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:54.786854 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:54.786857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:54.787047 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:54.787167 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:54.787174 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:54.787908 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:54.787912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:54.788488 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:54.788496 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:54.788919 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:54.788923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:54.789017 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:54.789480 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:54.789495 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:54.791108 (+ 1613us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":70,"threads_started":1}
W20260807 05:09:54.791358 17699 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:54.786287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:52105 (local address 127.8.16.126:42575)
0807 05:09:54.786415 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:54.786419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:54.786568 (+ 149us) server_negotiation.cc:407] Connection header received
0807 05:09:54.786695 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:54.786698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:54.786748 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:54.786813 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:54.787276 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:54.787802 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:54.788597 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:54.788819 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:54.788903 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:54.791251 (+ 2348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.66:52105: BlockingRecv error: recv got EOF from 127.8.16.66:52105 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 07 05:09:54 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:54 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:54.971568 17701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:54.966537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.67:52907)
0807 05:09:54.966797 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:54.966812 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:54.966909 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:54.967156 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:54.967158 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:54.967333 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:54.967427 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:54.967432 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:54.968157 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:54.968160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:54.968741 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:54.968750 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:54.969183 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:54.969186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:54.969255 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:54.969733 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:54.969745 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:54.971441 (+ 1696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":100,"threads_started":1}
W20260807 05:09:54.971683 17699 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:54.966582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:52907 (local address 127.8.16.126:42575)
0807 05:09:54.966693 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:54.966696 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:54.966872 (+ 176us) server_negotiation.cc:407] Connection header received
0807 05:09:54.967018 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:54.967021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:54.967070 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:54.967145 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:54.967506 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:54.968032 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:54.968847 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:54.969066 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:54.969151 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:54.971576 (+ 2425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.67:52907: BlockingRecv error: recv got EOF from 127.8.16.67:52907 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 07 05:09:55 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:55 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:55.607411 17702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:55.602302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.65:51501)
0807 05:09:55.602568 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:55.602580 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:55.602667 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:55.602882 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:55.602884 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:55.603057 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:55.603167 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:55.603173 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:55.603891 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:55.603893 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:55.604442 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:55.604449 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:55.604807 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:55.604810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:55.604868 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:55.605313 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:55.605323 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:55.607232 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":89,"threads_started":1}
W20260807 05:09:55.607478 17703 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:55.602376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:51501 (local address 127.8.16.126:42575)
0807 05:09:55.602626 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:55.602629 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:55.602642 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:09:55.602744 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:55.602746 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:55.602789 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:55.602861 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:55.603285 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:55.603816 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:55.604517 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:55.604725 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:55.604793 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:55.607367 (+ 2574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.65:51501: BlockingRecv error: recv got EOF from 127.8.16.65:51501 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":93,"threads_started":1}
Aug 07 05:09:55 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:55 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:55.796739 17704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:55.791903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.66:37859)
0807 05:09:55.792172 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:55.792186 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:55.792280 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:55.792478 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:55.792480 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:55.792664 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:55.792759 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:55.792764 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:55.793435 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:55.793437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:55.794002 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:55.794010 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:55.794375 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:55.794377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:55.794435 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:55.794861 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:55.794872 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:55.796552 (+ 1680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":100,"threads_started":1}
W20260807 05:09:55.796795 17703 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:55.791994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:37859 (local address 127.8.16.126:42575)
0807 05:09:55.792140 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:55.792161 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:55.792245 (+ 84us) server_negotiation.cc:407] Connection header received
0807 05:09:55.792348 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:55.792350 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:55.792393 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:55.792459 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:55.792838 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:55.793358 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:55.794084 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:55.794296 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:55.794367 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:55.796697 (+ 2330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.66:37859: BlockingRecv error: recv got EOF from 127.8.16.66:37859 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 05:09:55 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:55 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:55.977929 17706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:55.972377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.67:60247)
0807 05:09:55.972636 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:55.972650 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:55.972743 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:55.972984 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:55.972986 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:55.973169 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:55.973277 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:55.973282 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:55.974082 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:55.974084 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:55.974709 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:55.974721 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:55.975274 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:55.975277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:55.975348 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:55.975910 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:55.975923 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:55.977781 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":77,"threads_started":1}
W20260807 05:09:55.977991 17703 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:55.972564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:60247 (local address 127.8.16.126:42575)
0807 05:09:55.972723 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:55.972727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:55.972741 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:09:55.972825 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:55.972827 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:55.972872 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:55.972949 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:55.973433 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:55.973996 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:55.974831 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:55.975133 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:55.975260 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:55.977897 (+ 2637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.67:60247: BlockingRecv error: recv got EOF from 127.8.16.67:60247 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 07 05:09:56 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:56 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:56.613400 17708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:56.608127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.65:42555)
0807 05:09:56.608431 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:56.608446 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:56.608537 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:56.608749 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:56.608751 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:56.608939 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:56.609034 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:56.609039 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:56.609730 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:56.609732 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:56.610279 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:56.610288 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:56.610641 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:56.610644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:56.610722 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:56.611266 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:56.611278 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:56.613214 (+ 1936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":101,"threads_started":1}
W20260807 05:09:56.613504 17709 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:56.608284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:42555 (local address 127.8.16.126:42575)
0807 05:09:56.608484 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:56.608488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:56.608501 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:09:56.608612 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:56.608614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:56.608657 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:56.608727 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:56.609118 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:56.609657 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:56.610362 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:56.610569 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:56.610633 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:56.613361 (+ 2728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.65:42555: BlockingRecv error: recv got EOF from 127.8.16.65:42555 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":59,"threads_started":1}
Aug 07 05:09:56 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:56 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:56.802410 17710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:56.797487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.66:56283)
0807 05:09:56.797743 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:56.797758 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:56.797861 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:56.798053 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:56.798054 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:56.798239 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:56.798338 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:56.798343 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:56.798998 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:56.799000 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:56.799554 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:56.799561 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:56.799908 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:56.799910 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:56.799969 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:56.800414 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:56.800425 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:56.802258 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":81,"threads_started":1}
W20260807 05:09:56.802526 17709 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:56.797532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:56283 (local address 127.8.16.126:42575)
0807 05:09:56.797655 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:56.797658 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:56.797821 (+ 163us) server_negotiation.cc:407] Connection header received
0807 05:09:56.797922 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:56.797924 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:56.797966 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:56.798032 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:56.798416 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:56.798922 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:56.799630 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:56.799833 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:56.799896 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:56.802427 (+ 2531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.66:56283: BlockingRecv error: recv got EOF from 127.8.16.66:56283 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 07 05:09:56 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:56 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:56.984174 17711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:56.978795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.67:37663)
0807 05:09:56.979142 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:56.979157 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:56.979269 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:56.979496 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:56.979498 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:56.979683 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:56.979782 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:56.979787 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:56.980615 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:56.980619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:56.981246 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:56.981257 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:56.981817 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:56.981821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:56.981911 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:56.982376 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:56.982387 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:56.984024 (+ 1637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":107,"threads_started":1}
W20260807 05:09:56.984256 17709 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:56.978914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:37663 (local address 127.8.16.126:42575)
0807 05:09:56.979068 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:56.979072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:56.979225 (+ 153us) server_negotiation.cc:407] Connection header received
0807 05:09:56.979343 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:56.979345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:56.979394 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:56.979465 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:56.979869 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:56.980456 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:56.981368 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:56.981688 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:56.981795 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:56.984149 (+ 2354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.67:37663: BlockingRecv error: recv got EOF from 127.8.16.67:37663 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 05:09:57 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:57 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:57.623147 17712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:57.614123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.65:48831)
0807 05:09:57.614389 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:57.614402 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:57.614487 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:57.614706 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:57.614708 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:57.614885 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:57.614984 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:57.614989 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:57.615717 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:57.615719 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:57.616273 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:57.616279 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:57.616685 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:57.616687 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:57.616745 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:57.621155 (+ 4410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:57.621169 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:57.622988 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":76,"threads_started":1}
W20260807 05:09:57.623256 17713 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:57.614204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:48831 (local address 127.8.16.126:42575)
0807 05:09:57.614429 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:57.614433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:57.614445 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:09:57.614559 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:57.614561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:57.614607 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:57.614684 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:57.615095 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:57.615632 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:57.616368 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:57.616598 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:57.616673 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:57.623134 (+ 6461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.65:48831: BlockingRecv error: recv got EOF from 127.8.16.65:48831 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":69,"threads_started":1}
Aug 07 05:09:57 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:57 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:57.808454 17714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:57.803250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.66:33899)
0807 05:09:57.803503 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:57.803517 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:57.803631 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:57.803845 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:57.803847 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:57.804027 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:57.804127 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:57.804132 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:57.804875 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:57.804878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:57.805472 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:57.805483 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:57.805913 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:57.805916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:57.806003 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:57.806493 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:57.806504 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:57.808312 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":78,"threads_started":1}
W20260807 05:09:57.808655 17713 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:57.803342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:33899 (local address 127.8.16.126:42575)
0807 05:09:57.803489 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:57.803493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:57.803591 (+ 98us) server_negotiation.cc:407] Connection header received
0807 05:09:57.803703 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:57.803705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:57.803748 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:57.803818 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:57.804241 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:57.804773 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:57.805592 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:57.805811 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:57.805905 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:57.808458 (+ 2553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.66:33899: BlockingRecv error: recv got EOF from 127.8.16.66:33899 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 05:09:57 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:57 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:57.989970 17716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:57.984969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.67:45921)
0807 05:09:57.985248 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:57.985262 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:57.985363 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:57.985593 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:57.985595 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:57.985783 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:57.985882 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:57.985887 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:57.986586 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:57.986588 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:57.987153 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:57.987160 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:57.987559 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:57.987561 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:57.987619 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:57.988080 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:57.988091 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:57.989850 (+ 1759us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":97,"threads_started":1}
W20260807 05:09:57.990085 17713 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:57.985065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:45921 (local address 127.8.16.126:42575)
0807 05:09:57.985209 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:57.985212 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:57.985324 (+ 112us) server_negotiation.cc:407] Connection header received
0807 05:09:57.985432 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:57.985434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:57.985500 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:57.985567 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:57.985963 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:57.986508 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:57.987255 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:57.987471 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:57.987548 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:57.989984 (+ 2436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.67:45921: BlockingRecv error: recv got EOF from 127.8.16.67:45921 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 05:09:58 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:58 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:58.629571 17717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:58.624190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.65:51213)
0807 05:09:58.624477 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:58.624493 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:58.624623 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:58.624902 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:58.624905 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:58.625129 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:58.625221 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:58.625227 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:58.626025 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:58.626029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:58.626696 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:58.626706 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:58.627136 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:58.627139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:58.627219 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:58.627754 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:58.627766 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:58.629417 (+ 1651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":97,"threads_started":1}
W20260807 05:09:58.629690 17718 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:58.624240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:51213 (local address 127.8.16.126:42575)
0807 05:09:58.624477 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:58.624482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:58.624561 (+ 79us) server_negotiation.cc:407] Connection header received
0807 05:09:58.624717 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:58.624720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:58.624790 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:58.624867 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:58.625334 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:58.625924 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:58.626806 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:58.627037 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:58.627125 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:58.629576 (+ 2451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.65:51213: BlockingRecv error: recv got EOF from 127.8.16.65:51213 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":60,"threads_started":1}
Aug 07 05:09:58 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:58 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:58.814597 17719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:58.809204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.66:42273)
0807 05:09:58.809504 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:58.809518 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:58.809616 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:58.809831 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:58.809833 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:58.810019 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:58.810116 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:58.810121 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:58.810856 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:58.810858 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:58.811434 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:58.811442 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:58.811926 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:58.811929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:58.812010 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:58.812547 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:58.812562 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:58.814471 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":76,"threads_started":1}
W20260807 05:09:58.814723 17718 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:58.809368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:42273 (local address 127.8.16.126:42575)
0807 05:09:58.809508 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:58.809511 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:58.809594 (+ 83us) server_negotiation.cc:407] Connection header received
0807 05:09:58.809682 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:58.809684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:58.809728 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:58.809801 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:58.810218 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:58.810769 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:58.811551 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:58.811792 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:58.811875 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:58.814603 (+ 2728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.66:42273: BlockingRecv error: recv got EOF from 127.8.16.66:42273 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 07 05:09:58 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:58 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:58.996057 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:58.990733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.67:41219)
0807 05:09:58.991040 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:58.991055 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:58.991170 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:58.991413 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:58.991416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:58.991633 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:58.991732 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:58.991738 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:58.992518 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:58.992522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:58.993101 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:58.993109 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:58.993549 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:58.993552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:58.993646 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:58.994116 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:58.994128 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:58.995905 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":95,"threads_started":1}
W20260807 05:09:58.996178 17718 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:58.990840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:41219 (local address 127.8.16.126:42575)
0807 05:09:58.991024 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:58.991028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:58.991127 (+ 99us) server_negotiation.cc:407] Connection header received
0807 05:09:58.991257 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:58.991261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:58.991309 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:58.991377 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:58.991851 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:58.992406 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:58.993217 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:58.993427 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:58.993544 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:58.996045 (+ 2501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.67:41219: BlockingRecv error: recv got EOF from 127.8.16.67:41219 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 07 05:09:59 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:59 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:59.635449 17721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:59.630298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.65:56125)
0807 05:09:59.630568 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:59.630581 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:59.630684 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:59.630906 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:59.630909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:59.631094 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:59.631258 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:59.631265 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:59.632070 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:59.632074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:59.632719 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:59.632729 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:59.633129 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:59.633132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:59.633190 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:59.633664 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:59.633685 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:59.635334 (+ 1649us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":89,"threads_started":1}
W20260807 05:09:59.635582 17722 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:59.630381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:56125 (local address 127.8.16.126:42575)
0807 05:09:59.630600 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:59.630604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:59.630633 (+ 29us) server_negotiation.cc:407] Connection header received
0807 05:09:59.630758 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:59.630760 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:59.630802 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:59.630877 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:59.631369 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:59.631957 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:59.632819 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:59.633043 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:59.633119 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:59.635463 (+ 2344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.65:56125: BlockingRecv error: recv got EOF from 127.8.16.65:56125 (error 108)
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":60,"threads_started":1}
Aug 07 05:09:59 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:59 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:09:59.820757 17723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:59.815365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.66:56799)
0807 05:09:59.815617 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:59.815639 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:59.815740 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:59.815969 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:59.815972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:59.816165 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:59.816265 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:59.816271 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:59.817050 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:59.817054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:59.817865 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:59.817876 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:59.818308 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:59.818311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:59.818377 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:59.818851 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:59.818861 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:59.820627 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":75,"threads_started":1}
W20260807 05:09:59.820876 17722 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:59.815455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:56799 (local address 127.8.16.126:42575)
0807 05:09:59.815589 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:59.815592 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:59.815703 (+ 111us) server_negotiation.cc:407] Connection header received
0807 05:09:59.815810 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:59.815812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:59.815858 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:59.815925 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:59.816355 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:59.816922 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:59.817966 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:59.818213 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:59.818292 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:59.820772 (+ 2480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.66:56799: BlockingRecv error: recv got EOF from 127.8.16.66:56799 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 07 05:09:59 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:09:59 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:00.002010 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:59.996890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.67:51093)
0807 05:09:59.997157 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:59.997185 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:59.997286 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:59.997519 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:59.997522 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:59.997707 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:59.997804 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:59.997809 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:59.998490 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:59.998492 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:59.999038 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:59.999044 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:59.999462 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:59.999464 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:59.999528 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:00.000027 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:00.000039 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:00.001892 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":77,"threads_started":1}
W20260807 05:10:00.002125 17722 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:59.996981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:51093 (local address 127.8.16.126:42575)
0807 05:09:59.997108 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:59.997112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:59.997252 (+ 140us) server_negotiation.cc:407] Connection header received
0807 05:09:59.997357 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:59.997359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:59.997404 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:59.997496 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:59.997888 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:59.998406 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:59.999148 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:59.999367 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:59.999443 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:00.002020 (+ 2577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.67:51093: BlockingRecv error: recv got EOF from 127.8.16.67:51093 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 07 05:10:00 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:00 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:00.641291 17725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:00.636176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.65:34389)
0807 05:10:00.636481 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:00.636495 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:00.636595 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:00.636807 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:00.636809 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:00.636990 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:00.637083 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:00.637088 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:00.637812 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:00.637814 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:00.638361 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:00.638369 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:00.638795 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:00.638797 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:00.638858 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:00.639346 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:00.639358 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:00.641115 (+ 1757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":73,"threads_started":1}
W20260807 05:10:00.641374 17726 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:00.636242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:34389 (local address 127.8.16.126:42575)
0807 05:10:00.636501 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:00.636505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:00.636556 (+ 51us) server_negotiation.cc:407] Connection header received
0807 05:10:00.636665 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:00.636667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:00.636709 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:00.636784 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:00.637175 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:00.637734 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:00.638467 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:00.638704 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:00.638775 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:00.641260 (+ 2485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.65:34389: BlockingRecv error: recv got EOF from 127.8.16.65:34389 (error 108)
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":88,"threads_started":1}
Aug 07 05:10:00 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:00 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:00.827174 17727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:00.821538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.66:45039)
0807 05:10:00.821904 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:00.821917 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:00.822034 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:00.822305 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:00.822308 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:00.822512 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:00.822613 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:00.822619 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:00.823384 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:00.823393 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:00.823949 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:00.823957 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:00.824599 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:00.824603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:00.824694 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:00.825201 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:00.825212 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:00.827019 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":101,"threads_started":1}
W20260807 05:10:00.827276 17726 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:00.821635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:45039 (local address 127.8.16.126:42575)
0807 05:10:00.821835 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:00.821840 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:00.821991 (+ 151us) server_negotiation.cc:407] Connection header received
0807 05:10:00.822147 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:00.822150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:00.822201 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:00.822270 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:00.822732 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:00.823289 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:00.824142 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:00.824481 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:00.824570 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:00.827162 (+ 2592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.66:45039: BlockingRecv error: recv got EOF from 127.8.16.66:45039 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 07 05:10:01 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:01 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:01.008193 17728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:01.002868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.67:60559)
0807 05:10:01.003188 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:01.003201 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:01.003317 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:01.003566 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:01.003568 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:01.003779 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:01.003873 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:01.003879 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:01.004631 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:01.004635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:01.005178 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:01.005185 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:01.005658 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:01.005662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:01.005740 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:01.006258 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:01.006270 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:01.008038 (+ 1768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":86,"threads_started":1}
W20260807 05:10:01.008301 17726 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:01.002905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:60559 (local address 127.8.16.126:42575)
0807 05:10:01.003042 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:01.003047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:01.003282 (+ 235us) server_negotiation.cc:407] Connection header received
0807 05:10:01.003412 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:01.003415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:01.003462 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:01.003528 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:01.003990 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:01.004520 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:01.005287 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:01.005559 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:01.005649 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:01.008200 (+ 2551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.67:60559: BlockingRecv error: recv got EOF from 127.8.16.67:60559 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 07 05:10:01 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:01 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:01.647646 17729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:01.642184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.65:47729)
0807 05:10:01.642504 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:01.642518 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:01.642612 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:01.642860 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:01.642863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:01.643043 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:01.643140 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:01.643146 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:01.644089 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:01.644092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:01.644644 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:01.644652 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:01.645100 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:01.645104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:01.645189 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:01.645738 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:01.645751 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:01.647529 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":106,"threads_started":1}
W20260807 05:10:01.647804 17730 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:01.642343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:47729 (local address 127.8.16.126:42575)
0807 05:10:01.642584 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:01.642588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:01.642602 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:10:01.642692 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:01.642694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:01.642737 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:01.642811 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:01.643245 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:01.643982 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:01.644738 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:01.644996 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:01.645073 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:01.647668 (+ 2595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.65:47729: BlockingRecv error: recv got EOF from 127.8.16.65:47729 (error 108)
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":75,"threads_started":1}
Aug 07 05:10:01 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:01 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:01.833629 17731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:01.828046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.66:36681)
0807 05:10:01.828320 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:01.828336 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:01.828459 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:01.828729 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:01.828733 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:01.828975 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:01.829096 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:01.829106 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:01.830026 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:01.830029 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:01.830642 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:01.830653 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:01.831054 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:01.831057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:01.831124 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:01.831702 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:01.831716 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:01.833472 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":90,"threads_started":1}
W20260807 05:10:01.833736 17730 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:01.828157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:36681 (local address 127.8.16.126:42575)
0807 05:10:01.828318 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:01.828322 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:01.828423 (+ 101us) server_negotiation.cc:407] Connection header received
0807 05:10:01.828553 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:01.828557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:01.828606 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:01.828692 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:01.829219 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:01.829918 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:01.830739 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:01.830957 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:01.831027 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:01.833623 (+ 2596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.66:36681: BlockingRecv error: recv got EOF from 127.8.16.66:36681 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 05:10:02 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:02 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:02.014199 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:02.009027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.67:33067)
0807 05:10:02.009313 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:02.009326 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:02.009423 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:02.009668 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:02.009670 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:02.009855 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:02.009954 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:02.009960 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:02.010676 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:02.010678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:02.011230 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:02.011237 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:02.011598 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:02.011601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:02.011665 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:02.012137 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:02.012154 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:02.014046 (+ 1892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":98,"threads_started":1}
W20260807 05:10:02.014276 17730 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:02.009148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:33067 (local address 127.8.16.126:42575)
0807 05:10:02.009278 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:02.009282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:02.009387 (+ 105us) server_negotiation.cc:407] Connection header received
0807 05:10:02.009522 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:02.009524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:02.009569 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:02.009638 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:02.010052 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:02.010593 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:02.011310 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:02.011517 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:02.011582 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:02.014168 (+ 2586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.67:33067: BlockingRecv error: recv got EOF from 127.8.16.67:33067 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 05:10:02 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:02 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:02.657915 17733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:02.648415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.65:42905)
0807 05:10:02.648694 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:02.648709 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:02.648814 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:02.649058 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:02.649060 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:02.649266 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:02.649372 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:02.649378 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:02.650183 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:02.650186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:02.650762 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:02.650771 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:02.651243 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:02.651247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:02.651324 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:02.655804 (+ 4480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:02.655818 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:02.657792 (+ 1974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":74,"threads_started":1}
W20260807 05:10:02.658087 17734 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:02.648537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:42905 (local address 127.8.16.126:42575)
0807 05:10:02.648819 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:02.648823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:02.648837 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:10:02.648891 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:02.648894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:02.648936 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:02.649013 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:02.649510 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:02.650055 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:02.650877 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:02.651117 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:02.651200 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:02.657946 (+ 6746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.65:42905: BlockingRecv error: recv got EOF from 127.8.16.65:42905 (error 108)
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":112,"threads_started":1}
Aug 07 05:10:02 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:02 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:02.839861 17735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:02.834390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.66:52405)
0807 05:10:02.834669 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:02.834682 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:02.834780 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:02.835004 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:02.835006 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:02.835196 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:02.835292 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:02.835297 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:02.836065 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:02.836067 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:02.836646 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:02.836653 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:02.837138 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:02.837142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:02.837216 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:02.837752 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:02.837765 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:02.839691 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":83,"threads_started":1}
W20260807 05:10:02.839949 17734 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:02.834491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:52405 (local address 127.8.16.126:42575)
0807 05:10:02.834623 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:02.834626 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:02.834772 (+ 146us) server_negotiation.cc:407] Connection header received
0807 05:10:02.834846 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:02.834848 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:02.834896 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:02.834977 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:02.835417 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:02.835968 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:02.836763 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:02.837004 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:02.837104 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:02.839843 (+ 2739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.66:52405: BlockingRecv error: recv got EOF from 127.8.16.66:52405 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 07 05:10:03 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:03 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:03.020762 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:03.015131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.67:47599)
0807 05:10:03.015420 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:03.015436 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:03.015559 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:03.015883 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:03.015886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:03.016119 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:03.016238 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:03.016247 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:03.017055 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:03.017058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:03.017690 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:03.017699 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:03.018092 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:03.018095 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:03.018157 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:03.018633 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:03.018645 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:03.020590 (+ 1945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":87,"threads_started":1}
W20260807 05:10:03.020879 17734 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:03.015222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:47599 (local address 127.8.16.126:42575)
0807 05:10:03.015383 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:03.015388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:03.015517 (+ 129us) server_negotiation.cc:407] Connection header received
0807 05:10:03.015645 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:03.015647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:03.015701 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:03.015776 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:03.016352 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:03.016952 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:03.017778 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:03.017980 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:03.018045 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:03.020783 (+ 2738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.67:47599: BlockingRecv error: recv got EOF from 127.8.16.67:47599 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 07 05:10:03 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:03 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:03.663818 17737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:03.658665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.65:47641)
0807 05:10:03.659003 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:03.659018 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:03.659141 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:03.659363 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:03.659365 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:03.659551 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:03.659647 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:03.659652 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:03.660398 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:03.660401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:03.660967 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:03.660976 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:03.661394 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:03.661397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:03.661486 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:03.662009 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:03.662021 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:03.663673 (+ 1652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":77,"threads_started":1}
W20260807 05:10:03.663918 17738 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:03.658773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:47641 (local address 127.8.16.126:42575)
0807 05:10:03.659015 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:03.659019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:03.659087 (+ 68us) server_negotiation.cc:407] Connection header received
0807 05:10:03.659220 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:03.659222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:03.659264 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:03.659336 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:03.659744 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:03.660286 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:03.661074 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:03.661306 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:03.661380 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:03.663806 (+ 2426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.65:47641: BlockingRecv error: recv got EOF from 127.8.16.65:47641 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":87,"threads_started":1}
Aug 07 05:10:03 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:03 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:03.846127 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:03.840699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.66:46231)
0807 05:10:03.841014 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:03.841029 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:03.841137 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:03.841414 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:03.841416 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:03.841640 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:03.841743 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:03.841749 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:03.842515 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:03.842518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:03.843088 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:03.843096 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:03.843503 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:03.843506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:03.843573 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:03.844084 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:03.844096 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:03.845997 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":101,"threads_started":1}
W20260807 05:10:03.846261 17738 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:03.840811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:46231 (local address 127.8.16.126:42575)
0807 05:10:03.840966 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:03.840972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:03.841094 (+ 122us) server_negotiation.cc:407] Connection header received
0807 05:10:03.841255 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:03.841257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:03.841305 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:03.841372 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:03.841881 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:03.842412 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:03.843202 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:03.843408 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:03.843477 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:03.846133 (+ 2656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.66:46231: BlockingRecv error: recv got EOF from 127.8.16.66:46231 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 05:10:04 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:04 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:04.027336 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:04.021608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.67:58213)
0807 05:10:04.021906 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:04.021920 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:04.022019 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:04.022302 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:04.022304 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:04.022487 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:04.022588 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:04.022594 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:04.023533 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:04.023537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:04.024235 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:04.024249 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:04.024776 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:04.024779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:04.024848 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:04.025328 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:04.025339 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:04.027215 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":102,"threads_started":1}
W20260807 05:10:04.027527 17738 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:04.021778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:58213 (local address 127.8.16.126:42575)
0807 05:10:04.021938 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:04.021942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:04.021990 (+ 48us) server_negotiation.cc:407] Connection header received
0807 05:10:04.022143 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:04.022146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:04.022197 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:04.022277 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:04.022784 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:04.023427 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:04.024343 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:04.024642 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:04.024731 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:04.027394 (+ 2663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.67:58213: BlockingRecv error: recv got EOF from 127.8.16.67:58213 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 05:10:04 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:04 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:04.669840 17741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:04.664601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.65:41425)
0807 05:10:04.664881 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:04.664896 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:04.664986 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:04.665218 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:04.665220 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:04.665397 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:04.665534 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:04.665541 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:04.666285 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:04.666287 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:04.666846 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:04.666853 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:04.667301 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:04.667303 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:04.667359 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:04.667825 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:04.667836 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:04.669711 (+ 1875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":89,"threads_started":1}
W20260807 05:10:04.670024 17742 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:04.664701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:41425 (local address 127.8.16.126:42575)
0807 05:10:04.664955 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:04.664958 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:04.664971 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:10:04.665065 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:04.665068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:04.665117 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:04.665192 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:04.665628 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:04.666201 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:04.666973 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:04.667207 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:04.667307 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:04.669873 (+ 2566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.65:41425: BlockingRecv error: recv got EOF from 127.8.16.65:41425 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":59,"threads_started":1}
Aug 07 05:10:04 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:04 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:04.852490 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:04.846976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.66:58759)
0807 05:10:04.847297 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:04.847313 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:04.847457 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:04.847715 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:04.847718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:04.847915 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:04.848033 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:04.848042 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:04.848931 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:04.848933 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:04.849534 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:04.849543 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:04.849987 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:04.849990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:04.850064 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:04.850592 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:04.850604 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:04.852373 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":116,"threads_started":1}
W20260807 05:10:04.852655 17742 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:04.847121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:58759 (local address 127.8.16.126:42575)
0807 05:10:04.847299 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:04.847304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:04.847388 (+ 84us) server_negotiation.cc:407] Connection header received
0807 05:10:04.847550 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:04.847553 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:04.847601 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:04.847669 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:04.848131 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:04.848819 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:04.849632 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:04.849854 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:04.849925 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:04.852527 (+ 2602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.66:58759: BlockingRecv error: recv got EOF from 127.8.16.66:58759 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 07 05:10:05 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:05 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:05.033257 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:05.028102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.67:57691)
0807 05:10:05.028386 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:05.028402 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:05.028510 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:05.028720 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:05.028722 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:05.028908 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:05.029008 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:05.029013 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:05.029745 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:05.029749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:05.030320 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:05.030330 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:05.030688 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:05.030690 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:05.030748 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:05.031271 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:05.031283 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:05.033119 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":97,"threads_started":1}
W20260807 05:10:05.033344 17742 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:05.028156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:57691 (local address 127.8.16.126:42575)
0807 05:10:05.028301 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:05.028304 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:05.028464 (+ 160us) server_negotiation.cc:407] Connection header received
0807 05:10:05.028580 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:05.028583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:05.028628 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:05.028695 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:05.029091 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:05.029631 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:05.030407 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:05.030612 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:05.030677 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:05.033243 (+ 2566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.67:57691: BlockingRecv error: recv got EOF from 127.8.16.67:57691 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 07 05:10:05 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:05 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:05.675686 17745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:05.670640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.65:43291)
0807 05:10:05.670903 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:05.670917 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:05.671009 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:05.671238 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:05.671240 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:05.671423 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:05.671523 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:05.671528 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:05.672233 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:05.672235 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:05.672792 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:05.672800 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:05.673165 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:05.673167 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:05.673225 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:05.673779 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:05.673791 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:05.675529 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":79,"threads_started":1}
W20260807 05:10:05.675820 17746 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:05.670727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:43291 (local address 127.8.16.126:42575)
0807 05:10:05.670978 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:05.670982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:05.670996 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:10:05.671085 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:05.671087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:05.671131 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:05.671210 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:05.671611 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:05.672152 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:05.672874 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:05.673086 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:05.673154 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:05.675683 (+ 2529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.65:43291: BlockingRecv error: recv got EOF from 127.8.16.65:43291 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":74,"threads_started":1}
Aug 07 05:10:05 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:05 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:05.858708 17748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:05.853308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.66:36661)
0807 05:10:05.853601 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:05.853615 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:05.853733 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:05.853987 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:05.853990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:05.854203 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:05.854304 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:05.854310 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:05.855107 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:05.855111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:05.855754 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:05.855764 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:05.856169 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:05.856172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:05.856228 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:05.856730 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:05.856741 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:05.858583 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":87,"threads_started":1}
W20260807 05:10:05.858840 17746 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:05.853390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:36661 (local address 127.8.16.126:42575)
0807 05:10:05.853558 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:05.853561 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:05.853687 (+ 126us) server_negotiation.cc:407] Connection header received
0807 05:10:05.853819 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:05.853823 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:05.853878 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:05.853962 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:05.854447 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:05.854990 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:05.855842 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:05.856083 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:05.856162 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:05.858720 (+ 2558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.66:36661: BlockingRecv error: recv got EOF from 127.8.16.66:36661 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 05:10:06 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:06 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:06.039335 17749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:06.034048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.67:49031)
0807 05:10:06.034308 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:06.034322 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:06.034420 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:06.034632 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:06.034634 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:06.034817 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:06.034942 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:06.034947 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:06.035626 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:06.035628 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:06.036175 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:06.036182 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:06.036544 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:06.036547 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:06.036621 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:06.037065 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:06.037076 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:06.039181 (+ 2105us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":76,"threads_started":1}
W20260807 05:10:06.039433 17746 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:06.034138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:49031 (local address 127.8.16.126:42575)
0807 05:10:06.034310 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:06.034314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:06.034385 (+ 71us) server_negotiation.cc:407] Connection header received
0807 05:10:06.034491 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:06.034493 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:06.034540 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:06.034611 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:06.035025 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:06.035545 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:06.036260 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:06.036468 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:06.036534 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:06.039330 (+ 2796us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.67:49031: BlockingRecv error: recv got EOF from 127.8.16.67:49031 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 07 05:10:06 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:06 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:06.681772 17750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:06.676528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.65:49563)
0807 05:10:06.676806 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:06.676821 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:06.676917 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:06.677135 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:06.677137 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:06.677337 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:06.677432 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:06.677437 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:06.678239 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:06.678242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:06.678796 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:06.678802 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:06.679247 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:06.679249 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:06.679307 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:06.679794 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:06.679806 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:06.681649 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":79,"threads_started":1}
W20260807 05:10:06.681980 17751 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:06.676631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:49563 (local address 127.8.16.126:42575)
0807 05:10:06.676899 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:06.676902 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:06.676916 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:10:06.676984 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:06.676986 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:06.677029 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:06.677106 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:06.677598 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:06.678141 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:06.678918 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:06.679156 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:06.679238 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:06.681823 (+ 2585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.65:49563: BlockingRecv error: recv got EOF from 127.8.16.65:49563 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":92,"threads_started":1}
Aug 07 05:10:06 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:06 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:06.865003 17752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:06.859506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.66:57065)
0807 05:10:06.859792 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:06.859805 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:06.859903 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:06.860152 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:06.860154 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:06.860338 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:06.860436 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:06.860442 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:06.861203 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:06.861207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:06.861913 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:06.861926 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:06.862483 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:06.862487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:06.862561 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:06.863095 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:06.863107 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:06.864872 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":94,"threads_started":1}
W20260807 05:10:06.865109 17751 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:06.859654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:57065 (local address 127.8.16.126:42575)
0807 05:10:06.859784 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:06.859788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:06.859906 (+ 118us) server_negotiation.cc:407] Connection header received
0807 05:10:06.859985 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:06.859987 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:06.860036 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:06.860127 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:06.860557 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:06.861099 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:06.862027 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:06.862362 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:06.862463 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:06.865006 (+ 2543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.66:57065: BlockingRecv error: recv got EOF from 127.8.16.66:57065 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 07 05:10:07 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:07 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:07.046085 17753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:07.040295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.67:48923)
0807 05:10:07.040598 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:07.040612 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:07.040742 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:07.040978 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:07.040981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:07.041193 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:07.041291 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:07.041296 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:07.042221 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:07.042225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:07.042800 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:07.042809 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:07.043257 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:07.043260 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:07.043343 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:07.043877 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:07.043888 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:07.045932 (+ 2044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":98,"threads_started":1}
W20260807 05:10:07.046306 17751 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:07.040389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:48923 (local address 127.8.16.126:42575)
0807 05:10:07.040520 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:07.040525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:07.040682 (+ 157us) server_negotiation.cc:407] Connection header received
0807 05:10:07.040831 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:07.040834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:07.040879 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:07.040943 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:07.041408 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:07.042107 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:07.042918 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:07.043146 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:07.043245 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:07.046162 (+ 2917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.67:48923: BlockingRecv error: recv got EOF from 127.8.16.67:48923 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 07 05:10:07 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:07 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:07.697593 17754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:07.682560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.65:49819)
0807 05:10:07.682869 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:07.682882 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:07.682973 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:07.683197 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:07.683199 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:07.683385 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:07.683482 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:07.683487 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:07.684188 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:07.684190 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:07.684790 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:07.684798 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:07.685173 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:07.685176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:07.685232 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:07.695545 (+ 10313us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:07.695559 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:07.697418 (+ 1859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":106,"threads_started":1}
W20260807 05:10:07.697784 17755 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:07.682638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:49819 (local address 127.8.16.126:42575)
0807 05:10:07.682935 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:07.682938 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:07.682952 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:10:07.683047 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:07.683049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:07.683093 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:07.683170 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:07.683565 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:07.684102 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:07.684875 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:07.685095 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:07.685165 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:07.697637 (+ 12472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.65:49819: BlockingRecv error: recv got EOF from 127.8.16.65:49819 (error 108)
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":76,"threads_started":1}
Aug 07 05:10:07 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:07 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:07.871102 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:07.865850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.66:59151)
0807 05:10:07.866162 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:07.866177 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:07.866296 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:07.866551 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:07.866554 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:07.866755 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:07.866851 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:07.866857 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:07.867590 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:07.867594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:07.868151 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:07.868158 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:07.868579 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:07.868583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:07.868660 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:07.869179 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:07.869190 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:07.870986 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":118,"threads_started":1}
W20260807 05:10:07.871228 17755 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:07.865880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:59151 (local address 127.8.16.126:42575)
0807 05:10:07.866032 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:07.866036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:07.866263 (+ 227us) server_negotiation.cc:407] Connection header received
0807 05:10:07.866377 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:07.866380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:07.866433 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:07.866505 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:07.866965 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:07.867489 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:07.868262 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:07.868479 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:07.868564 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:07.871124 (+ 2560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.66:59151: BlockingRecv error: recv got EOF from 127.8.16.66:59151 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 05:10:08 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:08 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.67@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:08.051903 17757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:08.046795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.67:45389)
0807 05:10:08.047068 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:08.047092 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:08.047201 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:08.047459 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:08.047462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:08.047650 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:08.047761 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:08.047767 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:08.048453 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:08.048455 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:08.049019 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:08.049027 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:08.049419 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:08.049422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:08.049524 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:08.049991 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:08.050003 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:08.051785 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":110,"threads_started":1}
W20260807 05:10:08.052048 17755 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:08.046882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:45389 (local address 127.8.16.126:42575)
0807 05:10:08.047023 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:08.047027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:08.047186 (+ 159us) server_negotiation.cc:407] Connection header received
0807 05:10:08.047283 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:08.047286 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:08.047334 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:08.047395 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:08.047854 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:08.048372 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:08.049097 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:08.049306 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:08.049371 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:08.051924 (+ 2553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.67:45389: BlockingRecv error: recv got EOF from 127.8.16.67:45389 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 07 05:10:08 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:08 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.65@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:08.703998 17758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:08.698629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.65:49407)
0807 05:10:08.698911 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:08.698926 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:08.699028 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:08.699320 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:08.699323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:08.699531 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:08.699641 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:08.699648 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:08.700451 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:08.700454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:08.701036 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:08.701045 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:08.701559 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:08.701562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:08.701631 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:08.702091 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:08.702101 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:08.703871 (+ 1770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":93,"threads_started":1}
W20260807 05:10:08.704140 17759 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:08.698798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:49407 (local address 127.8.16.126:42575)
0807 05:10:08.699030 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:08.699034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:08.699047 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:10:08.699127 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:08.699130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:08.699199 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:08.699289 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:08.699763 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:08.700313 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:08.701166 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:08.701401 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:08.701552 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:08.704005 (+ 2453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.65:49407: BlockingRecv error: recv got EOF from 127.8.16.65:49407 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":71,"threads_started":1}
Aug 07 05:10:08 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:10:08 dist-test-slave-49h3 krb5kdc[16643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.16.66@KRBTEST.COM for oryx/127.8.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:10:08.877190 17760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:08.872022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:42575 (local address 127.8.16.66:57527)
0807 05:10:08.872298 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:08.872312 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:08.872436 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:08.872745 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:08.872749 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:08.872975 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:08.873099 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:08.873107 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:08.873865 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:08.873867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:08.874428 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:08.874435 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:08.874809 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:08.874812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:08.874873 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:08.875346 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:08.875358 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:08.877078 (+ 1720us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.16.126:42575: Server oryx/127.8.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":77,"threads_started":1}
W20260807 05:10:08.877306 17759 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:08.872145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:57527 (local address 127.8.16.126:42575)
0807 05:10:08.872287 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:08.872290 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:08.872403 (+ 113us) server_negotiation.cc:407] Connection header received
0807 05:10:08.872533 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:08.872536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:08.872592 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:08.872691 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:08.873226 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:08.873784 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:08.874510 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:08.874727 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:08.874795 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:08.877203 (+ 2408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.16.66:57527: BlockingRecv error: recv got EOF from 127.8.16.66:57527 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 05:10:08.951539 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 17258
I20260807 05:10:08.958303 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 17419
I20260807 05:10:08.963840 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 17558
I20260807 05:10:08.969233 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 17183
2026-08-07T05:10:08Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17851 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T05:10:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:10:08Z Disabled control of system clock
I20260807 05:10:08.998993 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:37071
--webserver_interface=127.8.16.126
--webserver_port=0
--builtin_ntp_servers=127.8.16.84:34549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:37071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 05:10:09.107649 17764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:10:09.107928 17764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:10:09.107961 17764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:10:09.111867 17764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:10:09.111922 17764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:10:09.111935 17764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:10:09.111951 17764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:10:09.111964 17764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:10:09.116571 17764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:34549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:37071
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:37071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:10:09.117615 17764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:10:09.118929 17764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:10:09.127362 17773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:09.127372 17770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:10:09.127419 17764 server_base.cc:1034] running on GCE node
W20260807 05:10:09.127372 17771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:10:09.128141 17764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:10:09.129216 17764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:10:09.130376 17764 hybrid_clock.cc:648] HybridClock initialized: now 1786079409130359 us; error 30 us; skew 500 ppm
I20260807 05:10:09.132129 17764 webserver.cc:460] Webserver started at http://127.8.16.126:34911/ using document root <none> and password file <none>
I20260807 05:10:09.132706 17764 fs_manager.cc:359] Metadata directory not provided
I20260807 05:10:09.132772 17764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:10:09.132977 17764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:10:09.135058 17764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "a5156c9503494239938eba3f0e05b73b"
format_stamp: "Formatted at 2026-08-07 05:10:09 on dist-test-slave-49h3"
server_key: "bf7662086b5117d1fd4cba9bc79f7e1e"
server_key_iv: "5da5fd62f55e46f3436d1e7377f53f12"
server_key_version: "encryptionkey@0"
I20260807 05:10:09.135619 17764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "a5156c9503494239938eba3f0e05b73b"
format_stamp: "Formatted at 2026-08-07 05:10:09 on dist-test-slave-49h3"
server_key: "bf7662086b5117d1fd4cba9bc79f7e1e"
server_key_iv: "5da5fd62f55e46f3436d1e7377f53f12"
server_key_version: "encryptionkey@0"
I20260807 05:10:09.139228 17764 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:10:09.141597 17779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:09.142814 17764 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:10:09.142920 17764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "a5156c9503494239938eba3f0e05b73b"
format_stamp: "Formatted at 2026-08-07 05:10:09 on dist-test-slave-49h3"
server_key: "bf7662086b5117d1fd4cba9bc79f7e1e"
server_key_iv: "5da5fd62f55e46f3436d1e7377f53f12"
server_key_version: "encryptionkey@0"
I20260807 05:10:09.143034 17764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:10:09.155766 17764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:10:09.156442 17764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:10:09.156621 17764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:10:09.163332 17764 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:37071
I20260807 05:10:09.163350 17831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:37071 every 8 connection(s)
I20260807 05:10:09.164263 17764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:10:09.164671 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 17764
I20260807 05:10:09.164780 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:10:09.165128 8257 external_mini_cluster.cc:1428] Setting key 955c4822417b3dfbd76690b1edb55434
I20260807 05:10:09.167153 17832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:10:09.172384 17832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b: Bootstrap starting.
I20260807 05:10:09.173906 17835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:09.166453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33362 (local address 127.8.16.126:37071)
0807 05:10:09.167125 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:09.167147 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:09.167191 (+ 44us) server_negotiation.cc:407] Connection header received
0807 05:10:09.167855 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:09.167869 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:09.168252 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:09.168547 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:09.168952 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:09.169963 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:09.170811 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:09.171270 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:09.171412 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:09.172214 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:10:09.172218 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:10:09.172230 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:10:09.172625 (+ 395us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:10:09.172681 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:10:09.172701 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:10:09.172786 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:09.172918 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 05:10:09.173173 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":119,"threads_started":1}
I20260807 05:10:09.174748 17832 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b: Neither blocks nor log segments found. Creating new log.
I20260807 05:10:09.175465 17832 log.cc:824] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b: Log is configured to *not* fsync() on all Append() calls
I20260807 05:10:09.177313 17832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b: No bootstrap required, opened a new log
I20260807 05:10:09.180581 17832 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5156c9503494239938eba3f0e05b73b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37071 } }
I20260807 05:10:09.180801 17832 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:10:09.180850 17832 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5156c9503494239938eba3f0e05b73b, State: Initialized, Role: FOLLOWER
I20260807 05:10:09.181237 17832 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5156c9503494239938eba3f0e05b73b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37071 } }
I20260807 05:10:09.181356 17832 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:10:09.181416 17832 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:10:09.181573 17832 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:10:09.182503 17832 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5156c9503494239938eba3f0e05b73b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37071 } }
I20260807 05:10:09.182798 17832 leader_election.cc:302] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5156c9503494239938eba3f0e05b73b; no voters:
I20260807 05:10:09.183063 17832 leader_election.cc:288] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:10:09.183177 17837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:10:09.183367 17837 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b [term 1 LEADER]: Becoming Leader. State: Replica: a5156c9503494239938eba3f0e05b73b, State: Running, Role: LEADER
I20260807 05:10:09.183696 17837 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5156c9503494239938eba3f0e05b73b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37071 } }
I20260807 05:10:09.184186 17832 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:10:09.185215 17839 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5156c9503494239938eba3f0e05b73b. Latest consensus state: current_term: 1 leader_uuid: "a5156c9503494239938eba3f0e05b73b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5156c9503494239938eba3f0e05b73b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37071 } } }
I20260807 05:10:09.185356 17839 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b [sys.catalog]: This master's current role is: LEADER
I20260807 05:10:09.185581 17838 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5156c9503494239938eba3f0e05b73b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5156c9503494239938eba3f0e05b73b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37071 } } }
I20260807 05:10:09.185683 17838 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b [sys.catalog]: This master's current role is: LEADER
W20260807 05:10:09.188787 17857 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:10:09.188863 17857 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:10:09.188918 17850 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:10:09.189692 17850 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:10:09.195145 17850 catalog_manager.cc:1347] Generated new cluster ID: 1b43695cd5f74f3eb437820739c9f756
I20260807 05:10:09.195207 17850 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:10:09.212759 17850 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:10:09.213727 17850 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:10:09.224642 17850 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5156c9503494239938eba3f0e05b73b: Generated new TSK 0
I20260807 05:10:09.225339 17850 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:10:09.232558 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37071
--builtin_ntp_servers=127.8.16.84:34549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 05:10:09.342317 17860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:10:09.342571 17860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:10:09.342607 17860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:10:09.346644 17860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:10:09.346707 17860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:10:09.346781 17860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:10:09.351485 17860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:34549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:10:09.352512 17860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:10:09.353892 17860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:10:09.363049 17866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:09.363049 17867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:09.363044 17869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:10:09.363327 17860 server_base.cc:1034] running on GCE node
I20260807 05:10:09.363708 17860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:10:09.364300 17860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:10:09.365500 17860 hybrid_clock.cc:648] HybridClock initialized: now 1786079409365481 us; error 29 us; skew 500 ppm
I20260807 05:10:09.371244 17860 webserver.cc:460] Webserver started at http://127.8.16.65:34775/ using document root <none> and password file <none>
I20260807 05:10:09.371802 17860 fs_manager.cc:359] Metadata directory not provided
I20260807 05:10:09.371867 17860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:10:09.372076 17860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:10:09.374183 17860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "544b0653d1fc4c3da12af7ce62364a7e"
format_stamp: "Formatted at 2026-08-07 05:10:09 on dist-test-slave-49h3"
server_key: "b1648a485383ce018ff325b889d5db9a"
server_key_iv: "37d5699ce80dd72e994c9a1a3b1c7fb7"
server_key_version: "encryptionkey@0"
I20260807 05:10:09.374699 17860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "544b0653d1fc4c3da12af7ce62364a7e"
format_stamp: "Formatted at 2026-08-07 05:10:09 on dist-test-slave-49h3"
server_key: "b1648a485383ce018ff325b889d5db9a"
server_key_iv: "37d5699ce80dd72e994c9a1a3b1c7fb7"
server_key_version: "encryptionkey@0"
I20260807 05:10:09.378225 17860 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:10:09.380566 17875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:09.381763 17860 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:10:09.381860 17860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "544b0653d1fc4c3da12af7ce62364a7e"
format_stamp: "Formatted at 2026-08-07 05:10:09 on dist-test-slave-49h3"
server_key: "b1648a485383ce018ff325b889d5db9a"
server_key_iv: "37d5699ce80dd72e994c9a1a3b1c7fb7"
server_key_version: "encryptionkey@0"
I20260807 05:10:09.381956 17860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:10:09.414450 17860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:10:09.415405 17860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:10:09.415673 17860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:10:09.416433 17860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:10:09.417748 17860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:10:09.417814 17860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:09.417877 17860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:10:09.417949 17860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:09.428534 17860 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:36427
I20260807 05:10:09.428604 17992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:36427 every 8 connection(s)
I20260807 05:10:09.429574 17860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:10:09.429822 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 17860
I20260807 05:10:09.429930 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:10:09.430272 8257 external_mini_cluster.cc:1428] Setting key 9b4ea06279a9e42ba5d90f92a3fff1b0
I20260807 05:10:09.432567 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37071
--builtin_ntp_servers=127.8.16.84:34549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 05:10:09.440524 17835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:09.431416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:40465 (local address 127.8.16.126:37071)
0807 05:10:09.431541 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:09.431545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:09.432957 (+ 1412us) server_negotiation.cc:407] Connection header received
0807 05:10:09.434727 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:09.434730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:09.434793 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:09.434877 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:09.436055 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:09.436827 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:09.438507 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:09.438742 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:09.438831 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:09.440016 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:10:09.440019 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:10:09.440022 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:10:09.440037 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:10:09.440061 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:10:09.440068 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:10:09.440127 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:09.440301 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 05:10:09.440373 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 05:10:09.441579 17995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:09.431956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:37071 (local address 127.8.16.65:40465)
0807 05:10:09.432796 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:09.432852 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:09.434505 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:09.435102 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:09.435120 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:09.435230 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:09.435898 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:09.435916 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:09.436968 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:09.436972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:09.438391 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:09.438403 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:09.438888 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:09.438893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:09.439104 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:09.439737 (+ 633us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:10:09.439739 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:09.439795 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:10:09.439878 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:10:09.440160 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:10:09.440169 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:10:09.440175 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 05:10:09.440293 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 05:10:09.440576 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":170,"threads_started":1}
I20260807 05:10:09.442772 17993 heartbeater.cc:347] Connected to a master server at 127.8.16.126:37071
I20260807 05:10:09.443004 17993 heartbeater.cc:464] Registering TS with master...
I20260807 05:10:09.443511 17993 heartbeater.cc:511] Master 127.8.16.126:37071 requested a full tablet report, sending...
I20260807 05:10:09.445058 17796 ts_manager.cc:194] Registered new tserver with Master: 544b0653d1fc4c3da12af7ce62364a7e (127.8.16.65:36427)
I20260807 05:10:09.446575 17796 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.16.65:40465
W20260807 05:10:09.544396 17996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:10:09.544661 17996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:10:09.544706 17996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:10:09.548743 17996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:10:09.548812 17996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:10:09.548868 17996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:10:09.553550 17996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:34549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:10:09.554584 17996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:10:09.555908 17996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:10:09.565196 18002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:09.565186 18001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:10:09.565299 17996 server_base.cc:1034] running on GCE node
W20260807 05:10:09.565196 18004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:10:09.565889 17996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:10:09.566483 17996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:10:09.567647 17996 hybrid_clock.cc:648] HybridClock initialized: now 1786079409567623 us; error 33 us; skew 500 ppm
I20260807 05:10:09.569376 17996 webserver.cc:460] Webserver started at http://127.8.16.66:34387/ using document root <none> and password file <none>
I20260807 05:10:09.569944 17996 fs_manager.cc:359] Metadata directory not provided
I20260807 05:10:09.570007 17996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:10:09.570209 17996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:10:09.572240 17996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "b9e2cc4ca358442b90c51502828ef57c"
format_stamp: "Formatted at 2026-08-07 05:10:09 on dist-test-slave-49h3"
server_key: "4a29f7670f01d0f4b7ddd691084af7b0"
server_key_iv: "0e5ba583239cbb08a7682618e5789526"
server_key_version: "encryptionkey@0"
I20260807 05:10:09.572784 17996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "b9e2cc4ca358442b90c51502828ef57c"
format_stamp: "Formatted at 2026-08-07 05:10:09 on dist-test-slave-49h3"
server_key: "4a29f7670f01d0f4b7ddd691084af7b0"
server_key_iv: "0e5ba583239cbb08a7682618e5789526"
server_key_version: "encryptionkey@0"
I20260807 05:10:09.576306 17996 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:10:09.578685 18010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:09.579921 17996 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:10:09.580031 17996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "b9e2cc4ca358442b90c51502828ef57c"
format_stamp: "Formatted at 2026-08-07 05:10:09 on dist-test-slave-49h3"
server_key: "4a29f7670f01d0f4b7ddd691084af7b0"
server_key_iv: "0e5ba583239cbb08a7682618e5789526"
server_key_version: "encryptionkey@0"
I20260807 05:10:09.580129 17996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:10:09.590204 17996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:10:09.590848 17996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:10:09.591020 17996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:10:09.591571 17996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:10:09.592497 17996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:10:09.592555 17996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:09.592608 17996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:10:09.592643 17996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:09.600943 17996 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:45477
I20260807 05:10:09.600997 18128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:45477 every 8 connection(s)
I20260807 05:10:09.601905 17996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:10:09.610203 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 17996
I20260807 05:10:09.610329 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:10:09.610620 8257 external_mini_cluster.cc:1428] Setting key 6003dd4d252bfade9df7fcbb2260dd9a
I20260807 05:10:09.610793 17835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:09.603613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:45285 (local address 127.8.16.126:37071)
0807 05:10:09.603745 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:09.603748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:09.604884 (+ 1136us) server_negotiation.cc:407] Connection header received
0807 05:10:09.606287 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:09.606289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:09.606340 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:09.606415 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:09.607297 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:09.607820 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:09.608889 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:09.609108 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:09.609196 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:09.610278 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:10:09.610282 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:10:09.610285 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:10:09.610301 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:10:09.610326 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:10:09.610337 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:10:09.610407 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:09.610568 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 05:10:09.610643 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 05:10:09.611711 18131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:09.604026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:37071 (local address 127.8.16.66:45285)
0807 05:10:09.604731 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:09.604777 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:09.606081 (+ 1304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:09.606566 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:09.606572 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:09.606665 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:09.607164 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:09.607177 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:09.607908 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:09.607910 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:09.608772 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:09.608781 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:09.609242 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:09.609245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:09.609385 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:09.609978 (+ 593us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:10:09.609980 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:09.610035 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:10:09.610117 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:10:09.610423 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:10:09.610434 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:10:09.610440 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 05:10:09.610554 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 05:10:09.610822 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":131,"threads_started":1}
I20260807 05:10:09.612593 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37071
--builtin_ntp_servers=127.8.16.84:34549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 05:10:09.612774 18129 heartbeater.cc:347] Connected to a master server at 127.8.16.126:37071
I20260807 05:10:09.612977 18129 heartbeater.cc:464] Registering TS with master...
I20260807 05:10:09.613476 18129 heartbeater.cc:511] Master 127.8.16.126:37071 requested a full tablet report, sending...
I20260807 05:10:09.614482 17796 ts_manager.cc:194] Registered new tserver with Master: b9e2cc4ca358442b90c51502828ef57c (127.8.16.66:45477)
I20260807 05:10:09.615025 17796 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.16.66:45285
W20260807 05:10:09.722755 18132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:10:09.723022 18132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:10:09.723057 18132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:10:09.727322 18132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:10:09.727375 18132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:10:09.727432 18132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:10:09.732190 18132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:34549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:10:09.733223 18132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:10:09.734606 18132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:10:09.743747 18139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:09.743763 18138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:09.743763 18141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:10:09.744228 18132 server_base.cc:1034] running on GCE node
I20260807 05:10:09.744593 18132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:10:09.745182 18132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:10:09.746337 18132 hybrid_clock.cc:648] HybridClock initialized: now 1786079409746319 us; error 28 us; skew 500 ppm
I20260807 05:10:09.748121 18132 webserver.cc:460] Webserver started at http://127.8.16.67:33955/ using document root <none> and password file <none>
I20260807 05:10:09.748669 18132 fs_manager.cc:359] Metadata directory not provided
I20260807 05:10:09.748730 18132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:10:09.748953 18132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:10:09.751009 18132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "c9c24bdb07ec4f6c9114e86ec42b2135"
format_stamp: "Formatted at 2026-08-07 05:10:09 on dist-test-slave-49h3"
server_key: "bd40a52f5df339069f7deee718616ce0"
server_key_iv: "3d91a88b9b282ea98c6f026f5d4bdd72"
server_key_version: "encryptionkey@0"
I20260807 05:10:09.751619 18132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "c9c24bdb07ec4f6c9114e86ec42b2135"
format_stamp: "Formatted at 2026-08-07 05:10:09 on dist-test-slave-49h3"
server_key: "bd40a52f5df339069f7deee718616ce0"
server_key_iv: "3d91a88b9b282ea98c6f026f5d4bdd72"
server_key_version: "encryptionkey@0"
I20260807 05:10:09.755239 18132 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:10:09.757699 18148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:09.758885 18132 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:10:09.758996 18132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "c9c24bdb07ec4f6c9114e86ec42b2135"
format_stamp: "Formatted at 2026-08-07 05:10:09 on dist-test-slave-49h3"
server_key: "bd40a52f5df339069f7deee718616ce0"
server_key_iv: "3d91a88b9b282ea98c6f026f5d4bdd72"
server_key_version: "encryptionkey@0"
I20260807 05:10:09.759109 18132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:10:09.775398 18132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:10:09.776027 18132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:10:09.776188 18132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:10:09.776719 18132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:10:09.777652 18132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:10:09.777704 18132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:09.777742 18132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:10:09.777762 18132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:09.786216 18132 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:37441
I20260807 05:10:09.786273 18266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:37441 every 8 connection(s)
I20260807 05:10:09.787055 18132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:10:09.789692 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 18132
I20260807 05:10:09.789803 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:10:09.790122 8257 external_mini_cluster.cc:1428] Setting key 976a8f0577d9132cb557c4cd324b46ca
I20260807 05:10:09.795910 17835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:09.788774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:38789 (local address 127.8.16.126:37071)
0807 05:10:09.788904 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:09.788908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:09.790107 (+ 1199us) server_negotiation.cc:407] Connection header received
0807 05:10:09.791693 (+ 1586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:09.791697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:09.791748 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:09.791829 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:09.792727 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:09.793269 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:09.794369 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:09.794585 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:09.794661 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:09.795554 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:10:09.795557 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:10:09.795559 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:10:09.795572 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:10:09.795591 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:10:09.795599 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:10:09.795652 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:09.795757 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 05:10:09.795804 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:10:09.796656 18269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:09.789157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:37071 (local address 127.8.16.67:38789)
0807 05:10:09.789910 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:09.789968 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:09.791479 (+ 1511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:09.791967 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:09.791973 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:09.792058 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:09.792599 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:09.792612 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:09.793356 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:09.793358 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:09.794260 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:09.794269 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:09.794691 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:09.794694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:09.794822 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:09.795334 (+ 512us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:10:09.795335 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:09.795371 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:10:09.795428 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:10:09.795658 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:10:09.795663 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:10:09.795666 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 05:10:09.795745 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 05:10:09.795940 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":119,"threads_started":1}
I20260807 05:10:09.797669 18267 heartbeater.cc:347] Connected to a master server at 127.8.16.126:37071
I20260807 05:10:09.797891 18267 heartbeater.cc:464] Registering TS with master...
I20260807 05:10:09.798349 18267 heartbeater.cc:511] Master 127.8.16.126:37071 requested a full tablet report, sending...
I20260807 05:10:09.799261 17796 ts_manager.cc:194] Registered new tserver with Master: c9c24bdb07ec4f6c9114e86ec42b2135 (127.8.16.67:37441)
I20260807 05:10:09.799775 17796 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.16.67:38789
I20260807 05:10:09.803822 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 05:10:09.806459 17835 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:09.805673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33364 (local address 127.8.16.126:37071)
0807 05:10:09.805783 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:09.805787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:09.805961 (+ 174us) server_negotiation.cc:407] Connection header received
0807 05:10:09.806051 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:09.806053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:09.806099 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:09.806159 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:09.806344 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33364: BlockingRecv error: recv got EOF from 127.0.0.1:33364 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 05:10:09.806856 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 17860
I20260807 05:10:09.813833 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 17996
I20260807 05:10:09.821413 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 18132
I20260807 05:10:09.826924 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 17764
2026-08-07T05:10:09Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (855 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T05:10:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:10:09Z Disabled control of system clock
I20260807 05:10:09.854959 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:35589
--webserver_interface=127.8.16.126
--webserver_port=0
--builtin_ntp_servers=127.8.16.84:37417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:35589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 05:10:09.966192 18279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:10:09.966482 18279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:10:09.966521 18279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:10:09.970713 18279 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:10:09.970788 18279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:10:09.970801 18279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:10:09.970816 18279 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:10:09.970831 18279 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:10:09.975795 18279 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:37417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:35589
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:35589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:10:09.976980 18279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:10:09.978440 18279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:10:09.986691 18285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:09.986685 18288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:09.986685 18286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:10:09.986928 18279 server_base.cc:1034] running on GCE node
I20260807 05:10:09.987546 18279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:10:09.988579 18279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:10:09.989727 18279 hybrid_clock.cc:648] HybridClock initialized: now 1786079409989712 us; error 29 us; skew 500 ppm
I20260807 05:10:09.991469 18279 webserver.cc:460] Webserver started at http://127.8.16.126:35017/ using document root <none> and password file <none>
I20260807 05:10:09.992010 18279 fs_manager.cc:359] Metadata directory not provided
I20260807 05:10:09.992069 18279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:10:09.992255 18279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:10:09.994297 18279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "f2964117489c4d739214dfb858b2ab2b"
format_stamp: "Formatted at 2026-08-07 05:10:09 on dist-test-slave-49h3"
server_key: "bac92446044144294f5e7038423d67e0"
server_key_iv: "b829c55e7d6655823d26743d31ac6d68"
server_key_version: "encryptionkey@0"
I20260807 05:10:09.994824 18279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "f2964117489c4d739214dfb858b2ab2b"
format_stamp: "Formatted at 2026-08-07 05:10:09 on dist-test-slave-49h3"
server_key: "bac92446044144294f5e7038423d67e0"
server_key_iv: "b829c55e7d6655823d26743d31ac6d68"
server_key_version: "encryptionkey@0"
I20260807 05:10:09.998570 18279 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 05:10:10.000897 18294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:10.002122 18279 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:10:10.002235 18279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "f2964117489c4d739214dfb858b2ab2b"
format_stamp: "Formatted at 2026-08-07 05:10:09 on dist-test-slave-49h3"
server_key: "bac92446044144294f5e7038423d67e0"
server_key_iv: "b829c55e7d6655823d26743d31ac6d68"
server_key_version: "encryptionkey@0"
I20260807 05:10:10.002328 18279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:10:10.006847 18279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:10:10.007403 18279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:10:10.007550 18279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:10:10.013716 18279 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:35589
I20260807 05:10:10.013726 18349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:35589 every 8 connection(s)
I20260807 05:10:10.014591 18279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:10:10.017776 18350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:10:10.020627 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 18279
I20260807 05:10:10.020758 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:10:10.021075 8257 external_mini_cluster.cc:1428] Setting key 90e30e6c2e6b6e0365745a1268174dca
I20260807 05:10:10.023996 18350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2964117489c4d739214dfb858b2ab2b: Bootstrap starting.
I20260807 05:10:10.026672 18350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2964117489c4d739214dfb858b2ab2b: Neither blocks nor log segments found. Creating new log.
I20260807 05:10:10.026840 18353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:10.022587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48830 (local address 127.8.16.126:35589)
0807 05:10:10.023209 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:10.023226 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:10.023262 (+ 36us) server_negotiation.cc:407] Connection header received
0807 05:10:10.023901 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:10.023919 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:10.024307 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:10.024590 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:10.025481 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:10:10.025489 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:10:10.025504 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:10:10.025515 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:10:10.025576 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:10:10.025600 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:10:10.025719 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:10.025819 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 05:10:10.026093 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":107,"threads_started":1}
I20260807 05:10:10.027498 18350 log.cc:824] T 00000000000000000000000000000000 P f2964117489c4d739214dfb858b2ab2b: Log is configured to *not* fsync() on all Append() calls
I20260807 05:10:10.029523 18350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2964117489c4d739214dfb858b2ab2b: No bootstrap required, opened a new log
I20260807 05:10:10.032982 18350 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2964117489c4d739214dfb858b2ab2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2964117489c4d739214dfb858b2ab2b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35589 } }
I20260807 05:10:10.033233 18350 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2964117489c4d739214dfb858b2ab2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:10:10.033284 18350 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2964117489c4d739214dfb858b2ab2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2964117489c4d739214dfb858b2ab2b, State: Initialized, Role: FOLLOWER
I20260807 05:10:10.033731 18350 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2964117489c4d739214dfb858b2ab2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2964117489c4d739214dfb858b2ab2b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35589 } }
I20260807 05:10:10.033862 18350 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2964117489c4d739214dfb858b2ab2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:10:10.033924 18350 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2964117489c4d739214dfb858b2ab2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:10:10.034018 18350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2964117489c4d739214dfb858b2ab2b [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:10:10.034946 18350 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2964117489c4d739214dfb858b2ab2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2964117489c4d739214dfb858b2ab2b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35589 } }
I20260807 05:10:10.035288 18350 leader_election.cc:302] T 00000000000000000000000000000000 P f2964117489c4d739214dfb858b2ab2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2964117489c4d739214dfb858b2ab2b; no voters:
I20260807 05:10:10.035619 18350 leader_election.cc:288] T 00000000000000000000000000000000 P f2964117489c4d739214dfb858b2ab2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:10:10.035720 18355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2964117489c4d739214dfb858b2ab2b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:10:10.035902 18355 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2964117489c4d739214dfb858b2ab2b [term 1 LEADER]: Becoming Leader. State: Replica: f2964117489c4d739214dfb858b2ab2b, State: Running, Role: LEADER
I20260807 05:10:10.036206 18355 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2964117489c4d739214dfb858b2ab2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2964117489c4d739214dfb858b2ab2b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35589 } }
I20260807 05:10:10.036945 18350 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2964117489c4d739214dfb858b2ab2b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:10:10.037914 18356 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2964117489c4d739214dfb858b2ab2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2964117489c4d739214dfb858b2ab2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2964117489c4d739214dfb858b2ab2b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35589 } } }
I20260807 05:10:10.038045 18356 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2964117489c4d739214dfb858b2ab2b [sys.catalog]: This master's current role is: LEADER
I20260807 05:10:10.038321 18357 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2964117489c4d739214dfb858b2ab2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2964117489c4d739214dfb858b2ab2b. Latest consensus state: current_term: 1 leader_uuid: "f2964117489c4d739214dfb858b2ab2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2964117489c4d739214dfb858b2ab2b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35589 } } }
I20260807 05:10:10.038411 18357 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2964117489c4d739214dfb858b2ab2b [sys.catalog]: This master's current role is: LEADER
I20260807 05:10:10.038667 18365 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:10:10.041755 18365 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:10:10.047534 18365 catalog_manager.cc:1347] Generated new cluster ID: dba7908fbce34f84acf605a663a8c37d
I20260807 05:10:10.047621 18365 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:10:10.055526 18365 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:10:10.055893 18365 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:10:10.063329 18365 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2964117489c4d739214dfb858b2ab2b: Generated new TSK 0
I20260807 05:10:10.064216 18365 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:10:10.068636 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:35589
--builtin_ntp_servers=127.8.16.84:37417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 05:10:10.178267 18375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:10:10.178537 18375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:10:10.178594 18375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:10:10.182705 18375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:10:10.182770 18375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:10:10.182848 18375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:10:10.187656 18375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:37417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:35589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:10:10.188737 18375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:10:10.190177 18375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:10:10.199291 18383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:10.199301 18380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:10.199293 18381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:10:10.199642 18375 server_base.cc:1034] running on GCE node
I20260807 05:10:10.200007 18375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:10:10.200590 18375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:10:10.201740 18375 hybrid_clock.cc:648] HybridClock initialized: now 1786079410201723 us; error 30 us; skew 500 ppm
I20260807 05:10:10.203462 18375 webserver.cc:460] Webserver started at http://127.8.16.65:39563/ using document root <none> and password file <none>
I20260807 05:10:10.203998 18375 fs_manager.cc:359] Metadata directory not provided
I20260807 05:10:10.204066 18375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:10:10.204269 18375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:10:10.206393 18375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "023965e49bae4d249cae110eb2276d2a"
format_stamp: "Formatted at 2026-08-07 05:10:10 on dist-test-slave-49h3"
server_key: "ac2664cc32acc68b2663d2600dc39c71"
server_key_iv: "1f42f745910b4535ca921c4bf11b6742"
server_key_version: "encryptionkey@0"
I20260807 05:10:10.206907 18375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "023965e49bae4d249cae110eb2276d2a"
format_stamp: "Formatted at 2026-08-07 05:10:10 on dist-test-slave-49h3"
server_key: "ac2664cc32acc68b2663d2600dc39c71"
server_key_iv: "1f42f745910b4535ca921c4bf11b6742"
server_key_version: "encryptionkey@0"
I20260807 05:10:10.210353 18375 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:10:10.212641 18389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:10.213819 18375 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:10:10.213932 18375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "023965e49bae4d249cae110eb2276d2a"
format_stamp: "Formatted at 2026-08-07 05:10:10 on dist-test-slave-49h3"
server_key: "ac2664cc32acc68b2663d2600dc39c71"
server_key_iv: "1f42f745910b4535ca921c4bf11b6742"
server_key_version: "encryptionkey@0"
I20260807 05:10:10.214023 18375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:10:10.218299 18375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:10:10.218853 18375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:10:10.218998 18375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:10:10.219494 18375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:10:10.220328 18375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:10:10.220373 18375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:10.220422 18375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:10:10.220449 18375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:10.228542 18375 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:35871
I20260807 05:10:10.228576 18508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:35871 every 8 connection(s)
I20260807 05:10:10.229387 18375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:10:10.235215 18353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:10.231131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:46561 (local address 127.8.16.126:35589)
0807 05:10:10.231263 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:10.231267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:10.232298 (+ 1031us) server_negotiation.cc:407] Connection header received
0807 05:10:10.233603 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:10.233607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:10.233653 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:10.233730 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:10.234774 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:10:10.234778 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:10:10.234781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:10:10.234793 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:10:10.234818 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:10:10.234826 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:10:10.234906 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:10.235026 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 05:10:10.235071 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:10:10.235625 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 18375
I20260807 05:10:10.235741 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:10:10.236078 8257 external_mini_cluster.cc:1428] Setting key 860c4ee61886eca10c49f84a27e9b65b
I20260807 05:10:10.236263 18512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:10.231489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:35589 (local address 127.8.16.65:46561)
0807 05:10:10.232151 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:10.232193 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:10.233365 (+ 1172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:10.233896 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:10.233902 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:10.233985 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:10.234499 (+ 514us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:10:10.234501 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:10.234549 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:10:10.234619 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:10:10.234896 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:10:10.234904 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:10:10.234910 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 05:10:10.235058 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 05:10:10.235294 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":106,"threads_started":1}
I20260807 05:10:10.237471 18510 heartbeater.cc:347] Connected to a master server at 127.8.16.126:35589
I20260807 05:10:10.237705 18510 heartbeater.cc:464] Registering TS with master...
I20260807 05:10:10.238161 18510 heartbeater.cc:511] Master 127.8.16.126:35589 requested a full tablet report, sending...
I20260807 05:10:10.238297 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:35589
--builtin_ntp_servers=127.8.16.84:37417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 05:10:10.239704 18311 ts_manager.cc:194] Registered new tserver with Master: 023965e49bae4d249cae110eb2276d2a (127.8.16.65:35871)
W20260807 05:10:10.345705 18513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:10:10.345983 18513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:10:10.346021 18513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:10:10.349982 18513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:10:10.350034 18513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:10:10.350090 18513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:10:10.354684 18513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:37417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:35589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:10:10.355708 18513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:10:10.357017 18513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:10:10.365934 18522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:10.365937 18519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:10.365969 18520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:10:10.366323 18513 server_base.cc:1034] running on GCE node
I20260807 05:10:10.366683 18513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:10:10.367254 18513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:10:10.368403 18513 hybrid_clock.cc:648] HybridClock initialized: now 1786079410368385 us; error 29 us; skew 500 ppm
I20260807 05:10:10.370209 18513 webserver.cc:460] Webserver started at http://127.8.16.66:45457/ using document root <none> and password file <none>
I20260807 05:10:10.370766 18513 fs_manager.cc:359] Metadata directory not provided
I20260807 05:10:10.370827 18513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:10:10.371022 18513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:10:10.373054 18513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "4f2006712f694ab5ac5dec935ebb1667"
format_stamp: "Formatted at 2026-08-07 05:10:10 on dist-test-slave-49h3"
server_key: "89a762bfa94a38658475dbdfb60f4e92"
server_key_iv: "7710188af017d1db895b63e85e730ab2"
server_key_version: "encryptionkey@0"
I20260807 05:10:10.373605 18513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "4f2006712f694ab5ac5dec935ebb1667"
format_stamp: "Formatted at 2026-08-07 05:10:10 on dist-test-slave-49h3"
server_key: "89a762bfa94a38658475dbdfb60f4e92"
server_key_iv: "7710188af017d1db895b63e85e730ab2"
server_key_version: "encryptionkey@0"
I20260807 05:10:10.377128 18513 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:10:10.379599 18528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:10.380743 18513 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 05:10:10.380844 18513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "4f2006712f694ab5ac5dec935ebb1667"
format_stamp: "Formatted at 2026-08-07 05:10:10 on dist-test-slave-49h3"
server_key: "89a762bfa94a38658475dbdfb60f4e92"
server_key_iv: "7710188af017d1db895b63e85e730ab2"
server_key_version: "encryptionkey@0"
I20260807 05:10:10.380951 18513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:10:10.385281 18513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:10:10.385874 18513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:10:10.386022 18513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:10:10.386533 18513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:10:10.387388 18513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:10:10.387434 18513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:10.387472 18513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:10:10.387501 18513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:10.395733 18513 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:46319
I20260807 05:10:10.395793 18651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:46319 every 8 connection(s)
I20260807 05:10:10.396622 18513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:10:10.402925 18353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:10.398472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:48025 (local address 127.8.16.126:35589)
0807 05:10:10.399026 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:10.399032 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:10.399852 (+ 820us) server_negotiation.cc:407] Connection header received
0807 05:10:10.401432 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:10.401436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:10.401516 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:10.401592 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:10.402580 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:10:10.402583 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:10:10.402585 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:10:10.402596 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:10:10.402616 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:10:10.402624 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:10:10.402704 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:10.402781 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 05:10:10.402816 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435}
I20260807 05:10:10.403779 18654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:10.398902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:35589 (local address 127.8.16.66:48025)
0807 05:10:10.399680 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:10.399736 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:10.401164 (+ 1428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:10.401768 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:10.401775 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:10.401861 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:10.402346 (+ 485us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:10:10.402347 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:10.402386 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:10:10.402450 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:10:10.402686 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:10:10.402690 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:10:10.402694 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 05:10:10.402811 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 05:10:10.402995 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":125,"threads_started":1}
I20260807 05:10:10.404790 18652 heartbeater.cc:347] Connected to a master server at 127.8.16.126:35589
I20260807 05:10:10.405002 18652 heartbeater.cc:464] Registering TS with master...
I20260807 05:10:10.405252 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 18513
I20260807 05:10:10.405376 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:10:10.405505 18652 heartbeater.cc:511] Master 127.8.16.126:35589 requested a full tablet report, sending...
I20260807 05:10:10.405766 8257 external_mini_cluster.cc:1428] Setting key a38d48958360124fae5ff1f59c2564b8
I20260807 05:10:10.406626 18311 ts_manager.cc:194] Registered new tserver with Master: 4f2006712f694ab5ac5dec935ebb1667 (127.8.16.66:46319)
I20260807 05:10:10.407943 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:35589
--builtin_ntp_servers=127.8.16.84:37417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 05:10:10.517328 18655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:10:10.517664 18655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:10:10.517760 18655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:10:10.521751 18655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:10:10.521804 18655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:10:10.521880 18655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:10:10.526567 18655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:37417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:35589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:10:10.527647 18655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:10:10.529063 18655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:10:10.538092 18660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:10.538177 18664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:10.538092 18661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:10:10.538192 18655 server_base.cc:1034] running on GCE node
I20260807 05:10:10.538854 18655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:10:10.539417 18655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:10:10.540565 18655 hybrid_clock.cc:648] HybridClock initialized: now 1786079410540547 us; error 29 us; skew 500 ppm
I20260807 05:10:10.542333 18655 webserver.cc:460] Webserver started at http://127.8.16.67:46429/ using document root <none> and password file <none>
I20260807 05:10:10.542886 18655 fs_manager.cc:359] Metadata directory not provided
I20260807 05:10:10.542948 18655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:10:10.543139 18655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:10:10.545089 18655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "8da8a3e31e0d45a79fd597cd348b4ca9"
format_stamp: "Formatted at 2026-08-07 05:10:10 on dist-test-slave-49h3"
server_key: "f8a75325a8d9ee506c734b927ec0d6aa"
server_key_iv: "e53b09c4bcd9862f7b15b73869dd8cc4"
server_key_version: "encryptionkey@0"
I20260807 05:10:10.545639 18655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "8da8a3e31e0d45a79fd597cd348b4ca9"
format_stamp: "Formatted at 2026-08-07 05:10:10 on dist-test-slave-49h3"
server_key: "f8a75325a8d9ee506c734b927ec0d6aa"
server_key_iv: "e53b09c4bcd9862f7b15b73869dd8cc4"
server_key_version: "encryptionkey@0"
I20260807 05:10:10.548959 18655 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260807 05:10:10.551307 18671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:10.552414 18655 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:10:10.552516 18655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "8da8a3e31e0d45a79fd597cd348b4ca9"
format_stamp: "Formatted at 2026-08-07 05:10:10 on dist-test-slave-49h3"
server_key: "f8a75325a8d9ee506c734b927ec0d6aa"
server_key_iv: "e53b09c4bcd9862f7b15b73869dd8cc4"
server_key_version: "encryptionkey@0"
I20260807 05:10:10.552605 18655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:10:10.556825 18655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:10:10.557374 18655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:10:10.557552 18655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:10:10.558061 18655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:10:10.558910 18655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:10:10.558956 18655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:10.559005 18655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:10:10.559031 18655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:10.567742 18655 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:36369
I20260807 05:10:10.567764 18790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:36369 every 8 connection(s)
I20260807 05:10:10.568598 18655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:10:10.574350 18353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:10.570339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:50799 (local address 127.8.16.126:35589)
0807 05:10:10.570507 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:10.570512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:10.571560 (+ 1048us) server_negotiation.cc:407] Connection header received
0807 05:10:10.572796 (+ 1236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:10.572798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:10.572835 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:10.572906 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:10.573949 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:10:10.573952 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:10:10.573954 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:10:10.573965 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:10:10.573985 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:10:10.573993 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:10:10.574066 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:10.574165 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 05:10:10.574213 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 05:10:10.574395 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 18655
I20260807 05:10:10.574496 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:10:10.574795 8257 external_mini_cluster.cc:1428] Setting key d28d790f82f3c47a465961b854eafc80
I20260807 05:10:10.575239 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:10.570756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:35589 (local address 127.8.16.67:50799)
0807 05:10:10.571414 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:10.571455 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:10.572619 (+ 1164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:10.573046 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:10.573053 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:10.573140 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:10.573694 (+ 554us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:10:10.573695 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:10.573737 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:10:10.573804 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:10:10.574056 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:10:10.574061 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:10:10.574064 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 05:10:10.574191 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 05:10:10.574379 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":102,"threads_started":1}
I20260807 05:10:10.576228 18791 heartbeater.cc:347] Connected to a master server at 127.8.16.126:35589
I20260807 05:10:10.576453 18791 heartbeater.cc:464] Registering TS with master...
I20260807 05:10:10.576884 18791 heartbeater.cc:511] Master 127.8.16.126:35589 requested a full tablet report, sending...
I20260807 05:10:10.577768 18311 ts_manager.cc:194] Registered new tserver with Master: 8da8a3e31e0d45a79fd597cd348b4ca9 (127.8.16.67:36369)
I20260807 05:10:10.588282 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 05:10:10.590849 18353 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:10.590116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48834 (local address 127.8.16.126:35589)
0807 05:10:10.590228 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:10.590231 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:10.590388 (+ 157us) server_negotiation.cc:407] Connection header received
0807 05:10:10.590481 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:10.590483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:10.590513 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:10.590571 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:10.590755 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48834: BlockingRecv error: recv got EOF from 127.0.0.1:48834 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 05:10:10.591035 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 18375
I20260807 05:10:10.601035 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 18513
I20260807 05:10:10.606972 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 18655
I20260807 05:10:10.613057 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 18279
2026-08-07T05:10:10Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (786 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:10:10 dist-test-slave-49h3 krb5kdc[18801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:10:10 dist-test-slave-49h3 krb5kdc[18801](info): set up 2 sockets
Aug 07 05:10:10 dist-test-slave-49h3 krb5kdc[18801](info): commencing operation
krb5kdc: starting...
W20260807 05:10:11.586216 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.953s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:10:11 dist-test-slave-49h3 krb5kdc[18801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:10:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:10:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:10:11.702395 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:37279
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:37279
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:10:11.809855 18817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:10:11.810111 18817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:10:11.810158 18817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:10:11.814057 18817 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:10:11.814113 18817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:10:11.814137 18817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:10:11.814152 18817 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:10:11.814165 18817 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:10:11.819324 18817 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:37279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:37279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:10:11.820299 18817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:10:11.821228 18817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:10:11.829159 18822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:11.829170 18823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:10:11.829216 18817 server_base.cc:1034] running on GCE node
W20260807 05:10:11.829169 18825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:10:11.830019 18817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:10:11.831068 18817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:10:11.832244 18817 hybrid_clock.cc:648] HybridClock initialized: now 1786079411832225 us; error 30 us; skew 500 ppm
Aug 07 05:10:11 dist-test-slave-49h3 krb5kdc[18801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079411, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:10:11.834821 18817 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:10:11.835646 18817 webserver.cc:460] Webserver started at http://127.8.16.126:35511/ using document root <none> and password file <none>
I20260807 05:10:11.836151 18817 fs_manager.cc:359] Metadata directory not provided
I20260807 05:10:11.836218 18817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:10:11.836411 18817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:10:11.838517 18817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "1de451c115244dbfba5d75b576f8a75f"
format_stamp: "Formatted at 2026-08-07 05:10:11 on dist-test-slave-49h3"
server_key: "7190558de155a0f6b5aed6f5055edaee"
server_key_iv: "782f2981453cef2e004aa16f42ca0a0a"
server_key_version: "encryptionkey@0"
I20260807 05:10:11.839074 18817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "1de451c115244dbfba5d75b576f8a75f"
format_stamp: "Formatted at 2026-08-07 05:10:11 on dist-test-slave-49h3"
server_key: "7190558de155a0f6b5aed6f5055edaee"
server_key_iv: "782f2981453cef2e004aa16f42ca0a0a"
server_key_version: "encryptionkey@0"
I20260807 05:10:11.842675 18817 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:10:11.844991 18832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:11.846215 18817 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:10:11.846338 18817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "1de451c115244dbfba5d75b576f8a75f"
format_stamp: "Formatted at 2026-08-07 05:10:11 on dist-test-slave-49h3"
server_key: "7190558de155a0f6b5aed6f5055edaee"
server_key_iv: "782f2981453cef2e004aa16f42ca0a0a"
server_key_version: "encryptionkey@0"
I20260807 05:10:11.846432 18817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:10:11.859416 18817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:10:11.862335 18817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:10:11.862517 18817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:10:11.869304 18817 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:37279
I20260807 05:10:11.869576 18888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:37279 every 8 connection(s)
I20260807 05:10:11.870317 18817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:10:11.872905 18889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:10:11.878052 18889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f: Bootstrap starting.
I20260807 05:10:11.878618 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 18817
I20260807 05:10:11.878755 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:10:11.879046 8257 external_mini_cluster.cc:1428] Setting key 5bba7fa7cb7f8adc9f84fcdf2f74f0c4
I20260807 05:10:11.880829 18889 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f: Neither blocks nor log segments found. Creating new log.
I20260807 05:10:11.881563 18889 log.cc:824] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f: Log is configured to *not* fsync() on all Append() calls
I20260807 05:10:11.883651 18889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f: No bootstrap required, opened a new log
I20260807 05:10:11.887861 18889 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1de451c115244dbfba5d75b576f8a75f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37279 } }
I20260807 05:10:11.888080 18889 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:10:11.888123 18889 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1de451c115244dbfba5d75b576f8a75f, State: Initialized, Role: FOLLOWER
Aug 07 05:10:11 dist-test-slave-49h3 krb5kdc[18801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:10:11.888576 18889 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1de451c115244dbfba5d75b576f8a75f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37279 } }
I20260807 05:10:11.888703 18889 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:10:11.888765 18889 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:10:11.888857 18889 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:10:11.889763 18889 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1de451c115244dbfba5d75b576f8a75f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37279 } }
I20260807 05:10:11.890067 18889 leader_election.cc:302] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1de451c115244dbfba5d75b576f8a75f; no voters:
I20260807 05:10:11.890381 18889 leader_election.cc:288] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:10:11.890455 18895 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:10:11.890652 18895 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f [term 1 LEADER]: Becoming Leader. State: Replica: 1de451c115244dbfba5d75b576f8a75f, State: Running, Role: LEADER
I20260807 05:10:11.890942 18895 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1de451c115244dbfba5d75b576f8a75f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37279 } }
I20260807 05:10:11.891664 18889 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:10:11.892495 18898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1de451c115244dbfba5d75b576f8a75f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1de451c115244dbfba5d75b576f8a75f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37279 } } }
I20260807 05:10:11.892628 18898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f [sys.catalog]: This master's current role is: LEADER
I20260807 05:10:11.892912 18902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1de451c115244dbfba5d75b576f8a75f. Latest consensus state: current_term: 1 leader_uuid: "1de451c115244dbfba5d75b576f8a75f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1de451c115244dbfba5d75b576f8a75f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 37279 } } }
I20260807 05:10:11.893000 18902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f [sys.catalog]: This master's current role is: LEADER
I20260807 05:10:11.893271 18909 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:10:11.894300 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:11.880366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35940 (local address 127.8.16.126:37279)
0807 05:10:11.880784 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:11.880800 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:11.880835 (+ 35us) server_negotiation.cc:407] Connection header received
0807 05:10:11.881537 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:11.881556 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:11.881905 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:11.882288 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:11.883126 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:11.884154 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:11.885345 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:11.885870 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:11.885996 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:11.888731 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:10:11.888738 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:10:11.888754 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:10:11.888789 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:10:11.890636 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:10:11.891143 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:10:11.891149 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:10:11.891155 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:10:11.891268 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:10:11.891534 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:10:11.891537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:10:11.891539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:10:11.892013 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:10:11.892879 (+ 866us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:11.893026 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 05:10:11.893333 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":128,"threads_started":1}
I20260807 05:10:11.896359 18909 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:10:11.901301 18909 catalog_manager.cc:1347] Generated new cluster ID: 59c413b5201d456c9fa9aaf0e9a22eb8
I20260807 05:10:11.901368 18909 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:10:11.912961 18909 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:10:11.914270 18909 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:10:11.920862 18909 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f: Generated new TSK 0
I20260807 05:10:11.921700 18909 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
I20260807 05:10:11.955875 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37279
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:10:12.064389 18922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:10:12.064643 18922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:10:12.064704 18922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:10:12.068673 18922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:10:12.068733 18922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:10:12.068813 18922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:10:12.073968 18922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:10:12.074958 18922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:10:12.075930 18922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:10:12.084972 18929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:12.084966 18931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:12.085006 18927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:10:12.085187 18922 server_base.cc:1034] running on GCE node
I20260807 05:10:12.085671 18922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:10:12.086299 18922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:10:12.087450 18922 hybrid_clock.cc:648] HybridClock initialized: now 1786079412087424 us; error 42 us; skew 500 ppm
Aug 07 05:10:12 dist-test-slave-49h3 krb5kdc[18801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079412, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:10:12.090188 18922 init.cc:377] Logged in from keytab as kudu/127.8.16.65@KRBTEST.COM (short username kudu)
I20260807 05:10:12.091099 18922 webserver.cc:460] Webserver started at http://127.8.16.65:43057/ using document root <none> and password file <none>
I20260807 05:10:12.091643 18922 fs_manager.cc:359] Metadata directory not provided
I20260807 05:10:12.091719 18922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:10:12.091924 18922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:10:12.094060 18922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "4a5b0d99b25047daa72f0011c499579f"
format_stamp: "Formatted at 2026-08-07 05:10:12 on dist-test-slave-49h3"
server_key: "5caced21e6581322b496e675702bde33"
server_key_iv: "8c3fb2ccb9cfcc0111c106b4643bd913"
server_key_version: "encryptionkey@0"
I20260807 05:10:12.094596 18922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "4a5b0d99b25047daa72f0011c499579f"
format_stamp: "Formatted at 2026-08-07 05:10:12 on dist-test-slave-49h3"
server_key: "5caced21e6581322b496e675702bde33"
server_key_iv: "8c3fb2ccb9cfcc0111c106b4643bd913"
server_key_version: "encryptionkey@0"
I20260807 05:10:12.098235 18922 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:10:12.100507 18938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:12.101737 18922 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:10:12.101852 18922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "4a5b0d99b25047daa72f0011c499579f"
format_stamp: "Formatted at 2026-08-07 05:10:12 on dist-test-slave-49h3"
server_key: "5caced21e6581322b496e675702bde33"
server_key_iv: "8c3fb2ccb9cfcc0111c106b4643bd913"
server_key_version: "encryptionkey@0"
I20260807 05:10:12.101966 18922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:10:12.121062 18922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:10:12.123906 18922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:10:12.124089 18922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:10:12.124668 18922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:10:12.125499 18922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:10:12.125547 18922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:12.125597 18922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:10:12.125626 18922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:12.134079 18922 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:43587
I20260807 05:10:12.134130 19055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:43587 every 8 connection(s)
I20260807 05:10:12.134943 18922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:10:12.142757 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 18922
I20260807 05:10:12.142887 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:10:12.143185 8257 external_mini_cluster.cc:1428] Setting key 7686c70bcc7239089ebccc5f5a01f419
Aug 07 05:10:12 dist-test-slave-49h3 krb5kdc[18801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079412, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:10:12.148252 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:12.136697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:35885 (local address 127.8.16.126:37279)
0807 05:10:12.136824 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:12.136827 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:12.137654 (+ 827us) server_negotiation.cc:407] Connection header received
0807 05:10:12.138947 (+ 1293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:12.138950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:12.139006 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:12.139078 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:12.140197 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.140736 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.141869 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.142103 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.142193 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:12.144877 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:10:12.144880 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:10:12.144882 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:10:12.144907 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:10:12.146224 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:10:12.146736 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:10:12.146739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:10:12.146741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:10:12.146834 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:10:12.147204 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:10:12.147207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:10:12.147209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:10:12.147431 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:10:12.147530 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:12.147963 (+ 433us) server_negotiation.cc:299] Negotiation successful
0807 05:10:12.148096 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 05:10:12.148998 19058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:12.137029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:37279 (local address 127.8.16.65:35885)
0807 05:10:12.137522 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:12.137564 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:12.138749 (+ 1185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:12.139251 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:12.139257 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:12.139595 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:12.140040 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:12.140052 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.140850 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.140853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:12.141760 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:12.141769 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.142242 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.142246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:12.142393 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:12.143047 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:12.143088 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:12.144721 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:10:12.146342 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:10:12.146348 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:10:12.146365 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:10:12.146649 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:10:12.146947 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:10:12.146950 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:10:12.146951 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:10:12.147095 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:10:12.147519 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:10:12.147524 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:10:12.147739 (+ 215us) client_negotiation.cc:769] Sending connection context
0807 05:10:12.147885 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 05:10:12.148125 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":121,"threads_started":1}
I20260807 05:10:12.150169 19056 heartbeater.cc:347] Connected to a master server at 127.8.16.126:37279
I20260807 05:10:12.150385 19056 heartbeater.cc:464] Registering TS with master...
I20260807 05:10:12.150895 19056 heartbeater.cc:511] Master 127.8.16.126:37279 requested a full tablet report, sending...
I20260807 05:10:12.152335 18850 ts_manager.cc:194] Registered new tserver with Master: 4a5b0d99b25047daa72f0011c499579f (127.8.16.65:43587)
WARNING: no policy specified for kudu/127.8.16.66@KRBTEST.COM; defaulting to no policy
I20260807 05:10:12.153534 18850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:35885
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
I20260807 05:10:12.168476 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37279
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:10:12.274617 19061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:10:12.274863 19061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:10:12.274904 19061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:10:12.278798 19061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:10:12.278854 19061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:10:12.278910 19061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:10:12.284000 19061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:10:12.284940 19061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:10:12.285914 19061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:10:12.294967 19072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:12.294987 19067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:12.294970 19068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:10:12.295174 19061 server_base.cc:1034] running on GCE node
I20260807 05:10:12.295634 19061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:10:12.296208 19061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:10:12.297358 19061 hybrid_clock.cc:648] HybridClock initialized: now 1786079412297341 us; error 28 us; skew 500 ppm
Aug 07 05:10:12 dist-test-slave-49h3 krb5kdc[18801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079412, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:10:12.299970 19061 init.cc:377] Logged in from keytab as kudu/127.8.16.66@KRBTEST.COM (short username kudu)
I20260807 05:10:12.300810 19061 webserver.cc:460] Webserver started at http://127.8.16.66:41571/ using document root <none> and password file <none>
I20260807 05:10:12.301323 19061 fs_manager.cc:359] Metadata directory not provided
I20260807 05:10:12.301383 19061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:10:12.301601 19061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:10:12.303560 19061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "33efc888d80a4de6986e5c405a5031d1"
format_stamp: "Formatted at 2026-08-07 05:10:12 on dist-test-slave-49h3"
server_key: "f4d257c16b46399419a79913aaaf8fa6"
server_key_iv: "417d416e227eca6ef93bbf96eda83675"
server_key_version: "encryptionkey@0"
I20260807 05:10:12.304077 19061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "33efc888d80a4de6986e5c405a5031d1"
format_stamp: "Formatted at 2026-08-07 05:10:12 on dist-test-slave-49h3"
server_key: "f4d257c16b46399419a79913aaaf8fa6"
server_key_iv: "417d416e227eca6ef93bbf96eda83675"
server_key_version: "encryptionkey@0"
I20260807 05:10:12.307667 19061 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260807 05:10:12.309975 19079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:12.311127 19061 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:10:12.311223 19061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "33efc888d80a4de6986e5c405a5031d1"
format_stamp: "Formatted at 2026-08-07 05:10:12 on dist-test-slave-49h3"
server_key: "f4d257c16b46399419a79913aaaf8fa6"
server_key_iv: "417d416e227eca6ef93bbf96eda83675"
server_key_version: "encryptionkey@0"
I20260807 05:10:12.311322 19061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:10:12.332859 19061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:10:12.335762 19061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:10:12.335937 19061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:10:12.336514 19061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:10:12.337327 19061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:10:12.337375 19061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:12.337427 19061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:10:12.337478 19061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:12.345711 19061 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:45591
I20260807 05:10:12.345738 19197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:45591 every 8 connection(s)
I20260807 05:10:12.346546 19061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:10:12.355159 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 19061
I20260807 05:10:12.355280 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
Aug 07 05:10:12 dist-test-slave-49h3 krb5kdc[18801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079412, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:10:12.355566 8257 external_mini_cluster.cc:1428] Setting key def87deb416c13be338db3398085a58c
I20260807 05:10:12.359282 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:12.348173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:44357 (local address 127.8.16.126:37279)
0807 05:10:12.348337 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:12.348342 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:12.349113 (+ 771us) server_negotiation.cc:407] Connection header received
0807 05:10:12.350475 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:12.350479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:12.350539 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:12.350620 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:12.351740 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.352271 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.353326 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.353577 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.353669 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:12.356021 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:10:12.356024 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:10:12.356026 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:10:12.356048 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:10:12.357389 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:10:12.357881 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:10:12.357885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:10:12.357887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:10:12.357964 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:10:12.358284 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:10:12.358287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:10:12.358289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:10:12.358464 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:10:12.358548 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:12.359007 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 05:10:12.359124 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 05:10:12.360145 19200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:12.348534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:37279 (local address 127.8.16.66:44357)
0807 05:10:12.348959 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:12.349000 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:12.350271 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:12.350799 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:12.350806 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:12.351162 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:12.351610 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:12.351624 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.352359 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.352361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:12.353225 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:12.353233 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.353733 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.353737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:12.353888 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:12.354375 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:12.354391 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:12.355856 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:10:12.357526 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:10:12.357532 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:10:12.357544 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:10:12.357790 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:10:12.358043 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:10:12.358045 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:10:12.358047 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:10:12.358197 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:10:12.358559 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:10:12.358566 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:10:12.358790 (+ 224us) client_negotiation.cc:769] Sending connection context
0807 05:10:12.358947 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 05:10:12.359268 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":117,"threads_started":1}
I20260807 05:10:12.361321 19198 heartbeater.cc:347] Connected to a master server at 127.8.16.126:37279
I20260807 05:10:12.361595 19198 heartbeater.cc:464] Registering TS with master...
I20260807 05:10:12.362142 19198 heartbeater.cc:511] Master 127.8.16.126:37279 requested a full tablet report, sending...
I20260807 05:10:12.363224 18850 ts_manager.cc:194] Registered new tserver with Master: 33efc888d80a4de6986e5c405a5031d1 (127.8.16.66:45591)
I20260807 05:10:12.363757 18850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:44357
WARNING: no policy specified for kudu/127.8.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
I20260807 05:10:12.382397 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37279
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:10:12.489557 19203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:10:12.489797 19203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:10:12.489837 19203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:10:12.493858 19203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:10:12.493922 19203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:10:12.493983 19203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:10:12.499017 19203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:37279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:10:12.499962 19203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:10:12.500900 19203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:10:12.510174 19209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:12.510179 19210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:12.510265 19212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:10:12.510469 19203 server_base.cc:1034] running on GCE node
I20260807 05:10:12.510928 19203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:10:12.511523 19203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:10:12.512677 19203 hybrid_clock.cc:648] HybridClock initialized: now 1786079412512663 us; error 29 us; skew 500 ppm
Aug 07 05:10:12 dist-test-slave-49h3 krb5kdc[18801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079412, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:10:12.515422 19203 init.cc:377] Logged in from keytab as kudu/127.8.16.67@KRBTEST.COM (short username kudu)
I20260807 05:10:12.516268 19203 webserver.cc:460] Webserver started at http://127.8.16.67:46693/ using document root <none> and password file <none>
I20260807 05:10:12.516780 19203 fs_manager.cc:359] Metadata directory not provided
I20260807 05:10:12.516847 19203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:10:12.517041 19203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:10:12.519093 19203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "19e0fd0b71a548dcb09127c6e5ac9c76"
format_stamp: "Formatted at 2026-08-07 05:10:12 on dist-test-slave-49h3"
server_key: "4564004485f27de7e46b5d8538245040"
server_key_iv: "7c42c8f999b1b1b8db44cd1166d96cd1"
server_key_version: "encryptionkey@0"
I20260807 05:10:12.519656 19203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "19e0fd0b71a548dcb09127c6e5ac9c76"
format_stamp: "Formatted at 2026-08-07 05:10:12 on dist-test-slave-49h3"
server_key: "4564004485f27de7e46b5d8538245040"
server_key_iv: "7c42c8f999b1b1b8db44cd1166d96cd1"
server_key_version: "encryptionkey@0"
I20260807 05:10:12.523273 19203 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 05:10:12.525594 19219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:12.526760 19203 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:10:12.526865 19203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "19e0fd0b71a548dcb09127c6e5ac9c76"
format_stamp: "Formatted at 2026-08-07 05:10:12 on dist-test-slave-49h3"
server_key: "4564004485f27de7e46b5d8538245040"
server_key_iv: "7c42c8f999b1b1b8db44cd1166d96cd1"
server_key_version: "encryptionkey@0"
I20260807 05:10:12.526957 19203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:10:12.541879 19203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:10:12.544888 19203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:10:12.545061 19203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:10:12.545642 19203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:10:12.546438 19203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:10:12.546494 19203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:12.546545 19203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:10:12.546571 19203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:12.555205 19203 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:40451
I20260807 05:10:12.555230 19338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:40451 every 8 connection(s)
I20260807 05:10:12.556030 19203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:10:12.558626 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 19203
I20260807 05:10:12.558729 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:10:12.558990 8257 external_mini_cluster.cc:1428] Setting key 6f4e2a6eafd857cdce4177af120e7a6a
Aug 07 05:10:12 dist-test-slave-49h3 krb5kdc[18801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079412, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:10:12.568691 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:12.557716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:60475 (local address 127.8.16.126:37279)
0807 05:10:12.557836 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:12.557840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:12.558583 (+ 743us) server_negotiation.cc:407] Connection header received
0807 05:10:12.560046 (+ 1463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:12.560050 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:12.560109 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:12.560181 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:12.561313 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.561903 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.563005 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.563234 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.563320 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:12.565659 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:10:12.565662 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:10:12.565665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:10:12.565691 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:10:12.566813 (+ 1122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:10:12.567270 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:10:12.567273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:10:12.567275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:10:12.567352 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:10:12.567655 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:10:12.567658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:10:12.567660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:10:12.567839 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:10:12.567924 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:12.568399 (+ 475us) server_negotiation.cc:299] Negotiation successful
0807 05:10:12.568518 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 05:10:12.569415 19342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:12.558020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:37279 (local address 127.8.16.67:60475)
0807 05:10:12.558441 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:12.558483 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:12.559836 (+ 1353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:12.560343 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:12.560350 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:12.560698 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:12.561172 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:12.561187 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.562020 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.562023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:12.562883 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:12.562892 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.563366 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.563370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:12.563528 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:12.564063 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:12.564078 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:12.565495 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:10:12.566915 (+ 1420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:10:12.566920 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:10:12.566934 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:10:12.567179 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:10:12.567431 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:10:12.567434 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:10:12.567436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:10:12.567562 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:10:12.567934 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:10:12.567941 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:10:12.568176 (+ 235us) client_negotiation.cc:769] Sending connection context
0807 05:10:12.568336 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 05:10:12.568638 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":115,"threads_started":1}
I20260807 05:10:12.570561 19339 heartbeater.cc:347] Connected to a master server at 127.8.16.126:37279
I20260807 05:10:12.570801 19339 heartbeater.cc:464] Registering TS with master...
I20260807 05:10:12.571305 19339 heartbeater.cc:511] Master 127.8.16.126:37279 requested a full tablet report, sending...
I20260807 05:10:12.572366 18850 ts_manager.cc:194] Registered new tserver with Master: 19e0fd0b71a548dcb09127c6e5ac9c76 (127.8.16.67:40451)
I20260807 05:10:12.572929 18850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:60475
I20260807 05:10:12.584443 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:10:12.593433 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:12.586344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35950 (local address 127.8.16.126:37279)
0807 05:10:12.586484 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:12.586489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:12.586637 (+ 148us) server_negotiation.cc:407] Connection header received
0807 05:10:12.586758 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:12.586761 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:12.586814 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:12.586879 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:12.587563 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.588086 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.588914 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.589128 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.589212 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:12.590521 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:10:12.590525 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:10:12.590527 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:10:12.590553 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:10:12.591750 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:10:12.592200 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:10:12.592203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:10:12.592205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:10:12.592285 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:10:12.592548 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:10:12.592552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:10:12.592553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:10:12.592775 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:10:12.592907 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:12.593159 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 05:10:12.593272 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 05:10:12.596271 18850 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:10:12.599233 18850 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:10:12.613276 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:12.607171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:43587 (local address 127.0.0.1:34260)
0807 05:10:12.608003 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:12.608066 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:12.608209 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:12.608821 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:12.608826 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:12.608850 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:10:12.609230 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:12.609239 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.610591 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.610594 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:12.611722 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:12.611732 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.612724 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.612728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:12.612832 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:12.612874 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 05:10:12.612990 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 05:10:12.613119 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"mutex_wait_us":202,"thread_start_us":64,"threads_started":1}
I20260807 05:10:12.613288 19354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:12.607307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34260 (local address 127.8.16.65:43587)
0807 05:10:12.608057 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:12.608074 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:12.608354 (+ 280us) server_negotiation.cc:407] Connection header received
0807 05:10:12.608443 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:12.608448 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:12.608655 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:12.608814 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:10:12.609349 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.610454 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.611843 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.612601 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.612725 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:12.612875 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:12.613031 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 05:10:12.613130 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":124,"threads_started":1}
I20260807 05:10:12.613641 19357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:12.608218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59396 (local address 127.8.16.66:45591)
0807 05:10:12.608655 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:12.608665 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:12.608685 (+ 20us) server_negotiation.cc:407] Connection header received
0807 05:10:12.608754 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:12.608758 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:12.608911 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:12.609042 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:10:12.609225 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.610618 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.612068 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.612829 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.612922 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:12.613035 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:12.613396 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 05:10:12.613507 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":142,"threads_started":1}
I20260807 05:10:12.613543 19353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:12.607286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:40451 (local address 127.0.0.1:48226)
0807 05:10:12.608208 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:12.608221 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:12.608312 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:12.608877 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:12.608879 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:12.608892 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:10:12.609068 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:12.609089 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.611265 (+ 2176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.611268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:12.612255 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:12.612263 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.613148 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.613152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:12.613238 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:12.613259 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 05:10:12.613304 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 05:10:12.613387 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":850,"mutex_wait_us":32,"thread_start_us":113,"threads_started":1}
I20260807 05:10:12.613955 19356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:12.607845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:45591 (local address 127.0.0.1:59396)
0807 05:10:12.608491 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:12.608502 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:12.608570 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:12.609023 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:12.609026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:12.609038 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:10:12.609116 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:12.609123 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.610786 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.610790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:12.611946 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:12.611957 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.613099 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.613103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:12.613202 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:12.613225 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:10:12.613293 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 05:10:12.613838 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":482,"thread_start_us":80,"threads_started":1}
I20260807 05:10:12.614414 19355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:12.607629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48226 (local address 127.8.16.67:40451)
0807 05:10:12.608439 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:12.608450 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:12.608471 (+ 21us) server_negotiation.cc:407] Connection header received
0807 05:10:12.608547 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:12.608552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:12.608736 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:12.609001 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:10:12.609176 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.611129 (+ 1953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.612361 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.613012 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.613634 (+ 622us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:12.614134 (+ 500us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:12.614188 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 05:10:12.614279 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":713,"thread_start_us":122,"threads_started":1}
I20260807 05:10:12.616768 18989 tablet_service.cc:1467] Processing CreateTablet for tablet c3a9cc2f1abd48619ce56576d69005bf (DEFAULT_TABLE table=test-table [id=33b210e545db40b0984c7c9bff28c08f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:10:12.616822 19130 tablet_service.cc:1467] Processing CreateTablet for tablet c3a9cc2f1abd48619ce56576d69005bf (DEFAULT_TABLE table=test-table [id=33b210e545db40b0984c7c9bff28c08f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:10:12.616823 19269 tablet_service.cc:1467] Processing CreateTablet for tablet c3a9cc2f1abd48619ce56576d69005bf (DEFAULT_TABLE table=test-table [id=33b210e545db40b0984c7c9bff28c08f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:10:12.617872 18989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3a9cc2f1abd48619ce56576d69005bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:10:12.617875 19130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3a9cc2f1abd48619ce56576d69005bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:10:12.617883 19269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3a9cc2f1abd48619ce56576d69005bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:10:12.623597 19358 tablet_bootstrap.cc:492] T c3a9cc2f1abd48619ce56576d69005bf P 4a5b0d99b25047daa72f0011c499579f: Bootstrap starting.
I20260807 05:10:12.625463 19359 tablet_bootstrap.cc:492] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1: Bootstrap starting.
I20260807 05:10:12.625883 19358 tablet_bootstrap.cc:654] T c3a9cc2f1abd48619ce56576d69005bf P 4a5b0d99b25047daa72f0011c499579f: Neither blocks nor log segments found. Creating new log.
I20260807 05:10:12.627019 19358 log.cc:824] T c3a9cc2f1abd48619ce56576d69005bf P 4a5b0d99b25047daa72f0011c499579f: Log is configured to *not* fsync() on all Append() calls
I20260807 05:10:12.627503 19359 tablet_bootstrap.cc:654] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1: Neither blocks nor log segments found. Creating new log.
I20260807 05:10:12.628147 19359 log.cc:824] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1: Log is configured to *not* fsync() on all Append() calls
I20260807 05:10:12.628727 19360 tablet_bootstrap.cc:492] T c3a9cc2f1abd48619ce56576d69005bf P 19e0fd0b71a548dcb09127c6e5ac9c76: Bootstrap starting.
I20260807 05:10:12.628883 19358 tablet_bootstrap.cc:492] T c3a9cc2f1abd48619ce56576d69005bf P 4a5b0d99b25047daa72f0011c499579f: No bootstrap required, opened a new log
I20260807 05:10:12.629060 19358 ts_tablet_manager.cc:1397] T c3a9cc2f1abd48619ce56576d69005bf P 4a5b0d99b25047daa72f0011c499579f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 05:10:12.630564 19360 tablet_bootstrap.cc:654] T c3a9cc2f1abd48619ce56576d69005bf P 19e0fd0b71a548dcb09127c6e5ac9c76: Neither blocks nor log segments found. Creating new log.
I20260807 05:10:12.631577 19360 log.cc:824] T c3a9cc2f1abd48619ce56576d69005bf P 19e0fd0b71a548dcb09127c6e5ac9c76: Log is configured to *not* fsync() on all Append() calls
I20260807 05:10:12.632174 19358 raft_consensus.cc:348] T c3a9cc2f1abd48619ce56576d69005bf P 4a5b0d99b25047daa72f0011c499579f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5b0d99b25047daa72f0011c499579f" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 43587 } } peers { permanent_uuid: "19e0fd0b71a548dcb09127c6e5ac9c76" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 40451 } } peers { permanent_uuid: "33efc888d80a4de6986e5c405a5031d1" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 45591 } }
I20260807 05:10:12.632447 19358 raft_consensus.cc:374] T c3a9cc2f1abd48619ce56576d69005bf P 4a5b0d99b25047daa72f0011c499579f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:10:12.632532 19358 raft_consensus.cc:729] T c3a9cc2f1abd48619ce56576d69005bf P 4a5b0d99b25047daa72f0011c499579f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a5b0d99b25047daa72f0011c499579f, State: Initialized, Role: FOLLOWER
I20260807 05:10:12.632936 19358 consensus_queue.cc:260] T c3a9cc2f1abd48619ce56576d69005bf P 4a5b0d99b25047daa72f0011c499579f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5b0d99b25047daa72f0011c499579f" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 43587 } } peers { permanent_uuid: "19e0fd0b71a548dcb09127c6e5ac9c76" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 40451 } } peers { permanent_uuid: "33efc888d80a4de6986e5c405a5031d1" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 45591 } }
I20260807 05:10:12.633128 19359 tablet_bootstrap.cc:492] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1: No bootstrap required, opened a new log
I20260807 05:10:12.633314 19359 ts_tablet_manager.cc:1397] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260807 05:10:12.633584 19056 heartbeater.cc:503] Master 127.8.16.126:37279 was elected leader, sending a full tablet report...
I20260807 05:10:12.633908 19358 ts_tablet_manager.cc:1428] T c3a9cc2f1abd48619ce56576d69005bf P 4a5b0d99b25047daa72f0011c499579f: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260807 05:10:12.635998 19057 tablet.cc:2369] T c3a9cc2f1abd48619ce56576d69005bf P 4a5b0d99b25047daa72f0011c499579f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:10:12.637821 19359 raft_consensus.cc:348] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5b0d99b25047daa72f0011c499579f" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 43587 } } peers { permanent_uuid: "19e0fd0b71a548dcb09127c6e5ac9c76" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 40451 } } peers { permanent_uuid: "33efc888d80a4de6986e5c405a5031d1" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 45591 } }
I20260807 05:10:12.638134 19359 raft_consensus.cc:374] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:10:12.638197 19359 raft_consensus.cc:729] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33efc888d80a4de6986e5c405a5031d1, State: Initialized, Role: FOLLOWER
I20260807 05:10:12.638595 19359 consensus_queue.cc:260] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5b0d99b25047daa72f0011c499579f" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 43587 } } peers { permanent_uuid: "19e0fd0b71a548dcb09127c6e5ac9c76" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 40451 } } peers { permanent_uuid: "33efc888d80a4de6986e5c405a5031d1" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 45591 } }
I20260807 05:10:12.638713 19360 tablet_bootstrap.cc:492] T c3a9cc2f1abd48619ce56576d69005bf P 19e0fd0b71a548dcb09127c6e5ac9c76: No bootstrap required, opened a new log
I20260807 05:10:12.638903 19360 ts_tablet_manager.cc:1397] T c3a9cc2f1abd48619ce56576d69005bf P 19e0fd0b71a548dcb09127c6e5ac9c76: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260807 05:10:12.639233 19198 heartbeater.cc:503] Master 127.8.16.126:37279 was elected leader, sending a full tablet report...
I20260807 05:10:12.639449 19359 ts_tablet_manager.cc:1428] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1: Time spent starting tablet: real 0.006s user 0.001s sys 0.003s
I20260807 05:10:12.642583 19360 raft_consensus.cc:348] T c3a9cc2f1abd48619ce56576d69005bf P 19e0fd0b71a548dcb09127c6e5ac9c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5b0d99b25047daa72f0011c499579f" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 43587 } } peers { permanent_uuid: "19e0fd0b71a548dcb09127c6e5ac9c76" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 40451 } } peers { permanent_uuid: "33efc888d80a4de6986e5c405a5031d1" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 45591 } }
I20260807 05:10:12.642843 19360 raft_consensus.cc:374] T c3a9cc2f1abd48619ce56576d69005bf P 19e0fd0b71a548dcb09127c6e5ac9c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:10:12.642901 19360 raft_consensus.cc:729] T c3a9cc2f1abd48619ce56576d69005bf P 19e0fd0b71a548dcb09127c6e5ac9c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19e0fd0b71a548dcb09127c6e5ac9c76, State: Initialized, Role: FOLLOWER
I20260807 05:10:12.643337 19360 consensus_queue.cc:260] T c3a9cc2f1abd48619ce56576d69005bf P 19e0fd0b71a548dcb09127c6e5ac9c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5b0d99b25047daa72f0011c499579f" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 43587 } } peers { permanent_uuid: "19e0fd0b71a548dcb09127c6e5ac9c76" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 40451 } } peers { permanent_uuid: "33efc888d80a4de6986e5c405a5031d1" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 45591 } }
I20260807 05:10:12.643992 19339 heartbeater.cc:503] Master 127.8.16.126:37279 was elected leader, sending a full tablet report...
I20260807 05:10:12.644328 19360 ts_tablet_manager.cc:1428] T c3a9cc2f1abd48619ce56576d69005bf P 19e0fd0b71a548dcb09127c6e5ac9c76: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260807 05:10:12.806924 19341 tablet.cc:2369] T c3a9cc2f1abd48619ce56576d69005bf P 19e0fd0b71a548dcb09127c6e5ac9c76: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 05:10:12.847648 19199 tablet.cc:2369] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:10:12.893322 19365 raft_consensus.cc:482] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:10:12.893543 19365 raft_consensus.cc:504] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5b0d99b25047daa72f0011c499579f" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 43587 } } peers { permanent_uuid: "19e0fd0b71a548dcb09127c6e5ac9c76" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 40451 } } peers { permanent_uuid: "33efc888d80a4de6986e5c405a5031d1" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 45591 } }
I20260807 05:10:12.894759 19365 leader_election.cc:288] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a5b0d99b25047daa72f0011c499579f (127.8.16.65:43587), 19e0fd0b71a548dcb09127c6e5ac9c76 (127.8.16.67:40451)
I20260807 05:10:12.898536 19354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:12.895103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:36959 (local address 127.8.16.65:43587)
0807 05:10:12.895303 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:12.895308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:12.895400 (+ 92us) server_negotiation.cc:407] Connection header received
0807 05:10:12.895543 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:12.895547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:12.895599 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:12.895658 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:10:12.895979 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.896489 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.897648 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.898024 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.898116 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:12.898186 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:12.898361 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 05:10:12.898426 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260807 05:10:12.898530 19355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:12.895142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:37569 (local address 127.8.16.67:40451)
0807 05:10:12.895358 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:12.895362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:12.895481 (+ 119us) server_negotiation.cc:407] Connection header received
0807 05:10:12.895606 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:12.895608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:12.895655 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:12.895729 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:10:12.896023 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.896506 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.897623 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.898014 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.898104 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:12.898168 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:12.898337 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 05:10:12.898404 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260807 05:10:12.898551 19368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:12.895031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:40451 (local address 127.8.16.66:37569)
0807 05:10:12.895379 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:12.895393 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:12.895504 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:12.895748 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:12.895751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:12.895766 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:10:12.895909 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:12.895913 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.896609 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.896613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:12.897509 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:12.897518 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.898124 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.898128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:12.898221 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:12.898244 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:10:12.898303 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 05:10:12.898371 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":57,"thread_start_us":79,"threads_started":1}
I20260807 05:10:12.898556 19367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:12.895021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:43587 (local address 127.8.16.66:36959)
0807 05:10:12.895301 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:12.895317 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:12.895437 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:12.895680 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:12.895683 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:12.895699 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:10:12.895869 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:12.895876 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.896592 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.896595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:12.897527 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:12.897537 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.898138 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.898142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:12.898247 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:12.898265 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 05:10:12.898335 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 05:10:12.898403 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":41,"thread_start_us":107,"threads_started":1}
I20260807 05:10:12.899163 19291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3a9cc2f1abd48619ce56576d69005bf" candidate_uuid: "33efc888d80a4de6986e5c405a5031d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19e0fd0b71a548dcb09127c6e5ac9c76" is_pre_election: true
I20260807 05:10:12.899206 19009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3a9cc2f1abd48619ce56576d69005bf" candidate_uuid: "33efc888d80a4de6986e5c405a5031d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a5b0d99b25047daa72f0011c499579f" is_pre_election: true
I20260807 05:10:12.899444 19291 raft_consensus.cc:2454] T c3a9cc2f1abd48619ce56576d69005bf P 19e0fd0b71a548dcb09127c6e5ac9c76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33efc888d80a4de6986e5c405a5031d1 in term 0.
I20260807 05:10:12.899472 19009 raft_consensus.cc:2454] T c3a9cc2f1abd48619ce56576d69005bf P 4a5b0d99b25047daa72f0011c499579f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33efc888d80a4de6986e5c405a5031d1 in term 0.
I20260807 05:10:12.899868 19083 leader_election.cc:302] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19e0fd0b71a548dcb09127c6e5ac9c76, 33efc888d80a4de6986e5c405a5031d1; no voters:
I20260807 05:10:12.900120 19365 raft_consensus.cc:2781] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:10:12.900197 19365 raft_consensus.cc:482] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:10:12.900228 19365 raft_consensus.cc:3037] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:10:12.901248 19365 raft_consensus.cc:504] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5b0d99b25047daa72f0011c499579f" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 43587 } } peers { permanent_uuid: "19e0fd0b71a548dcb09127c6e5ac9c76" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 40451 } } peers { permanent_uuid: "33efc888d80a4de6986e5c405a5031d1" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 45591 } }
I20260807 05:10:12.901649 19365 leader_election.cc:288] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1 [CANDIDATE]: Term 1 election: Requested vote from peers 4a5b0d99b25047daa72f0011c499579f (127.8.16.65:43587), 19e0fd0b71a548dcb09127c6e5ac9c76 (127.8.16.67:40451)
I20260807 05:10:12.901963 19009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3a9cc2f1abd48619ce56576d69005bf" candidate_uuid: "33efc888d80a4de6986e5c405a5031d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a5b0d99b25047daa72f0011c499579f"
I20260807 05:10:12.901968 19291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3a9cc2f1abd48619ce56576d69005bf" candidate_uuid: "33efc888d80a4de6986e5c405a5031d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19e0fd0b71a548dcb09127c6e5ac9c76"
I20260807 05:10:12.902071 19291 raft_consensus.cc:3037] T c3a9cc2f1abd48619ce56576d69005bf P 19e0fd0b71a548dcb09127c6e5ac9c76 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:10:12.902073 19009 raft_consensus.cc:3037] T c3a9cc2f1abd48619ce56576d69005bf P 4a5b0d99b25047daa72f0011c499579f [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:10:12.903040 19009 raft_consensus.cc:2454] T c3a9cc2f1abd48619ce56576d69005bf P 4a5b0d99b25047daa72f0011c499579f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33efc888d80a4de6986e5c405a5031d1 in term 1.
I20260807 05:10:12.903117 19291 raft_consensus.cc:2454] T c3a9cc2f1abd48619ce56576d69005bf P 19e0fd0b71a548dcb09127c6e5ac9c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33efc888d80a4de6986e5c405a5031d1 in term 1.
I20260807 05:10:12.903326 19082 leader_election.cc:302] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33efc888d80a4de6986e5c405a5031d1, 4a5b0d99b25047daa72f0011c499579f; no voters:
I20260807 05:10:12.903489 19365 raft_consensus.cc:2781] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:10:12.903723 19365 raft_consensus.cc:686] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1 [term 1 LEADER]: Becoming Leader. State: Replica: 33efc888d80a4de6986e5c405a5031d1, State: Running, Role: LEADER
I20260807 05:10:12.904038 19365 consensus_queue.cc:237] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5b0d99b25047daa72f0011c499579f" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 43587 } } peers { permanent_uuid: "19e0fd0b71a548dcb09127c6e5ac9c76" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 40451 } } peers { permanent_uuid: "33efc888d80a4de6986e5c405a5031d1" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 45591 } }
I20260807 05:10:12.907196 18850 catalog_manager.cc:5495] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33efc888d80a4de6986e5c405a5031d1 (127.8.16.66). New cstate: current_term: 1 leader_uuid: "33efc888d80a4de6986e5c405a5031d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5b0d99b25047daa72f0011c499579f" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 43587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19e0fd0b71a548dcb09127c6e5ac9c76" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33efc888d80a4de6986e5c405a5031d1" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260807 05:10:12.982494 19357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:12.979366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59402 (local address 127.8.16.66:45591)
0807 05:10:12.979498 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:12.979502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:12.979516 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:10:12.979590 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:12.979592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:12.979648 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:12.979733 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:10:12.979933 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.980434 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.981259 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:12.981481 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:12.981548 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:12.981770 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:10:12.982170 (+ 400us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:10:12.982253 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:12.982336 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 05:10:12.982386 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 05:10:12.987999 19009 raft_consensus.cc:1261] T c3a9cc2f1abd48619ce56576d69005bf P 4a5b0d99b25047daa72f0011c499579f [term 1 FOLLOWER]: Refusing update from remote peer 33efc888d80a4de6986e5c405a5031d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:10:12.987999 19291 raft_consensus.cc:1261] T c3a9cc2f1abd48619ce56576d69005bf P 19e0fd0b71a548dcb09127c6e5ac9c76 [term 1 FOLLOWER]: Refusing update from remote peer 33efc888d80a4de6986e5c405a5031d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:10:12.988699 19369 consensus_queue.cc:1035] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a5b0d99b25047daa72f0011c499579f" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 43587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:10:12.988901 19365 consensus_queue.cc:1035] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19e0fd0b71a548dcb09127c6e5ac9c76" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:10:12.997074 19374 mvcc.cc:205] Tried to move back new op lower bound from 7315781275592437760 to 7315781275257290752. Current Snapshot: MvccSnapshot[applied={T|T < 7315781275592437760}]
I20260807 05:10:13.004649 18849 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:10:13.004827 18849 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:10:13.007195 18849 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1de451c115244dbfba5d75b576f8a75f: Sending DeleteTablet for 3 replicas of tablet c3a9cc2f1abd48619ce56576d69005bf
I20260807 05:10:13.007982 18989 tablet_service.cc:1514] Processing DeleteTablet for tablet c3a9cc2f1abd48619ce56576d69005bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:10:12 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:34260
I20260807 05:10:13.008035 19269 tablet_service.cc:1514] Processing DeleteTablet for tablet c3a9cc2f1abd48619ce56576d69005bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:10:12 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:48226
I20260807 05:10:13.008136 19130 tablet_service.cc:1514] Processing DeleteTablet for tablet c3a9cc2f1abd48619ce56576d69005bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:10:12 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:59396
I20260807 05:10:13.008496 19383 tablet_replica.cc:330] T c3a9cc2f1abd48619ce56576d69005bf P 19e0fd0b71a548dcb09127c6e5ac9c76: stopping tablet replica
I20260807 05:10:13.008890 19383 raft_consensus.cc:2229] T c3a9cc2f1abd48619ce56576d69005bf P 19e0fd0b71a548dcb09127c6e5ac9c76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:10:13.008932 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 18922
I20260807 05:10:13.009743 19383 raft_consensus.cc:2258] T c3a9cc2f1abd48619ce56576d69005bf P 19e0fd0b71a548dcb09127c6e5ac9c76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:10:13.012009 19383 ts_tablet_manager.cc:1905] T c3a9cc2f1abd48619ce56576d69005bf P 19e0fd0b71a548dcb09127c6e5ac9c76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:10:13.012380 19385 tablet_replica.cc:330] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1: stopping tablet replica
I20260807 05:10:13.012657 19385 raft_consensus.cc:2229] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:10:13.012998 19385 raft_consensus.cc:2258] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:10:13.014468 19385 ts_tablet_manager.cc:1905] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:10:13.014830 19383 ts_tablet_manager.cc:1918] T c3a9cc2f1abd48619ce56576d69005bf P 19e0fd0b71a548dcb09127c6e5ac9c76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:10:13.014986 19383 log.cc:1196] T c3a9cc2f1abd48619ce56576d69005bf P 19e0fd0b71a548dcb09127c6e5ac9c76: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/c3a9cc2f1abd48619ce56576d69005bf
I20260807 05:10:13.015422 19383 ts_tablet_manager.cc:1939] T c3a9cc2f1abd48619ce56576d69005bf P 19e0fd0b71a548dcb09127c6e5ac9c76: Deleting consensus metadata
I20260807 05:10:13.016789 18837 catalog_manager.cc:4841] TS 19e0fd0b71a548dcb09127c6e5ac9c76 (127.8.16.67:40451): tablet c3a9cc2f1abd48619ce56576d69005bf (table test-table [id=33b210e545db40b0984c7c9bff28c08f]) successfully deleted
I20260807 05:10:13.017196 19385 ts_tablet_manager.cc:1918] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:10:13.017299 19385 log.cc:1196] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079354071073-8257-0/minicluster-data/ts-1/wal/wals/c3a9cc2f1abd48619ce56576d69005bf
I20260807 05:10:13.017645 19385 ts_tablet_manager.cc:1939] T c3a9cc2f1abd48619ce56576d69005bf P 33efc888d80a4de6986e5c405a5031d1: Deleting consensus metadata
I20260807 05:10:13.018672 18835 catalog_manager.cc:4841] TS 33efc888d80a4de6986e5c405a5031d1 (127.8.16.66:45591): tablet c3a9cc2f1abd48619ce56576d69005bf (table test-table [id=33b210e545db40b0984c7c9bff28c08f]) successfully deleted
I20260807 05:10:13.018935 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 19061
W20260807 05:10:13.018918 18835 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.16.65:43587 (error 108)
I20260807 05:10:13.020924 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:13.020581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:43587 (local address 127.0.0.1:34276)
0807 05:10:13.020772 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:13.020840 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.65:43587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260807 05:10:13.021167 18835 catalog_manager.cc:4584] TS 4a5b0d99b25047daa72f0011c499579f (127.8.16.65:43587): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c3a9cc2f1abd48619ce56576d69005bf: Network error: Client connection negotiation failed: client connection to 127.8.16.65:43587: connect: Connection refused (error 111)
I20260807 05:10:13.026480 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 19203
I20260807 05:10:13.031997 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 18817
2026-08-07T05:10:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2420 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:10:13 dist-test-slave-49h3 krb5kdc[19389](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:10:13 dist-test-slave-49h3 krb5kdc[19389](info): set up 2 sockets
Aug 07 05:10:13 dist-test-slave-49h3 krb5kdc[19389](info): commencing operation
krb5kdc: starting...
W20260807 05:10:14.030335 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.976s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:10:14 dist-test-slave-49h3 krb5kdc[19389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:10:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:10:14Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/rangeradmin_127.8.16.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/rangeradmin_127.8.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/rangerlookup_127.8.16.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/rangerlookup_127.8.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/HTTP_127.8.16.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/HTTP_127.8.16.84@KRBTEST.COM.keytab.
I20260807 05:10:14.185880 8257 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskg_K4zd/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/postgres -l logfile start
2026-08-07 05:10:16.375 UTC [19421] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 05:10:16.375 UTC [19421] LOG: listening on IPv4 address "127.8.16.84", port 36043
2026-08-07 05:10:16.378 UTC [19421] LOG: listening on Unix socket "/tmp/.s.PGSQL.36043"
2026-08-07 05:10:16.405 UTC [19425] LOG: database system was shut down at 2026-08-07 05:10:15 UTC
2026-08-07 05:10:16.408 UTC [19421] LOG: database system is ready to accept connections
I20260807 05:10:17.301919 8257 mini_postgres.cc:96] Postgres bound to 36043
2026-08-07 05:10:17.308 UTC [19439] FATAL: database "slave" does not exist
127.8.16.84:36043 - accepting connections
I20260807 05:10:17.309317 8257 mini_ranger.cc:161] Starting Ranger...
I20260807 05:10:17.323978 8257 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 05:10:17.712632 8257 mini_ranger.cc:87] Created ranger Postgres database
I20260807 05:10:17.712721 8257 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ranger-admin
2026-08-07 05:10:18,019 [I] DB FLAVOR :POSTGRES
2026-08-07 05:10:18,019 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 05:10:18,019 [I] Checking connection..
2026-08-07 05:10:18,019 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:36043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 05:10:18,270 [I] Checking connection passed.
2026-08-07 05:10:18,270 [I] --------- Verifying version history table ---------
2026-08-07 05:10:18,270 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:36043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 05:10:18,530 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 05:10:18,531 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:36043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 05:10:18,771 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 05:10:18,772 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 05:10:18,772 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:36043/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 05:10:19.017 UTC [19508] WARNING: there is no transaction in progress
2026-08-07 05:10:19,023 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 05:10:19,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:36043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 05:10:19,254 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 05:10:19,255 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 05:10:19,255 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:36043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 05:10:19,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:36043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 05:10:19,770 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:36043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-49h3.c.gcp-upstream.internal','N') ;"
2026-08-07 05:10:20,017 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:36043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 05:10:20,269 [I] Table x_portal_user does not exist in database ranger
2026-08-07 05:10:20,269 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:36043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 05:10:20,526 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 05:10:20,526 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:36043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 05:10:20,781 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:36043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 05:10:21,037 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 05:10:21,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:36043/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 05:10:21.912 UTC [19624] WARNING: there is no transaction in progress
2026-08-07 05:10:21.927 UTC [19624] WARNING: there is no transaction in progress
2026-08-07 05:10:21.943 UTC [19624] WARNING: there is no transaction in progress
2026-08-07 05:10:21.957 UTC [19624] WARNING: there is no transaction in progress
2026-08-07 05:10:21.971 UTC [19624] WARNING: there is no transaction in progress
2026-08-07 05:10:21.987 UTC [19624] WARNING: there is no transaction in progress
2026-08-07 05:10:22.039 UTC [19624] WARNING: there is no transaction in progress
2026-08-07 05:10:22.045 UTC [19624] WARNING: there is no transaction in progress
2026-08-07 05:10:22.057 UTC [19624] WARNING: there is no transaction in progress
2026-08-07 05:10:22.066 UTC [19624] WARNING: there is no transaction in progress
2026-08-07 05:10:22.075 UTC [19624] WARNING: there is no transaction in progress
2026-08-07 05:10:22.088 UTC [19624] WARNING: there is no transaction in progress
2026-08-07 05:10:22.097 UTC [19624] WARNING: there is no transaction in progress
2026-08-07 05:10:22.103 UTC [19624] WARNING: there is no transaction in progress
2026-08-07 05:10:22.109 UTC [19624] WARNING: there is no transaction in progress
2026-08-07 05:10:22.115 UTC [19624] WARNING: there is no transaction in progress
2026-08-07 05:10:22,651 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 05:10:22,651 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:36043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 05:10:22,874 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 05:10:22,874 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:36043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 05:10:23,110 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 05:10:23,111 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:36043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 05:10:23,369 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 05:10:23,369 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:36043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 05:10:23,627 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:36043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 05:10:23,885 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:36043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-49h3.c.gcp-upstream.internal';"
2026-08-07 05:10:24,139 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 05:10:24,140 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:36043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 05:10:24,397 [I] DB_PATCHES have already been applied
I20260807 05:10:24.402649 8257 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ranger-admin:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskg_K4zd/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 05:10:24.402719 8257 mini_ranger.cc:193] Using host: 127.8.16.84
I20260807 05:10:24.406574 8257 mini_ranger.cc:227] Ranger admin URL: http://127.8.16.84:43515
Aug 07, 2026 5:10:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 5:10:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 5:10:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 5:10:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 5:10:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 5:10:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 5:10:25 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 05:10:25.424044 8257 mini_ranger.cc:160] Time spent starting Ranger: real 8.115s user 0.000s sys 0.011s
Aug 07, 2026 5:10:44 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 5:10:44 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 5:10:44 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 5:10:44 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 5:10:45 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 05:10:45.872184 8257 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/rangerkms_127.8.16.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/rangerkms_127.8.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.16.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.16.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.16.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/HTTP_127.8.16.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.16.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/HTTP_127.8.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 05:10:45 dist-test-slave-49h3 krb5kdc[19389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079445, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 05:10:45.974786 8257 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 05:10:45.986012 8257 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 05:10:46.738641 8257 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 05:10:46.738732 8257 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ranger-kms
I20260807 05:10:46.738801 8257 mini_ranger_kms.cc:227] Using postgres at 127.8.16.84:36043
2026-08-07 05:10:46,962 [I] DB FLAVOR :POSTGRES
2026-08-07 05:10:46,962 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 05:10:46,962 [I] Checking connection
2026-08-07 05:10:46,963 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:36043/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 05:10:47,239 [I] connection success
2026-08-07 05:10:47,239 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 05:10:47,239 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 05:10:47,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:36043/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 05:10:47,482 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 05:10:47,482 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 05:10:47,482 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 05:10:47,482 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:36043/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 05:10:47,763 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 05:10:48.252424 8257 mini_ranger_kms.cc:310] Created kms service
I20260807 05:10:48.454993 8257 mini_ranger_kms.cc:326] Created kudu user
I20260807 05:10:48.496994 8257 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 05:10:48 dist-test-slave-49h3 krb5kdc[19389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079445, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.16.84@KRBTEST.COM
I20260807 05:10:48.859575 8257 mini_ranger_kms.cc:384] Added ranger policy
I20260807 05:10:48.859769 8257 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ranger-kms:/tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 05:10:48.859805 8257 mini_ranger_kms.cc:240] Using host: 127.8.16.84
I20260807 05:10:48.864494 8257 mini_ranger_kms.cc:276] Ranger KMS PID: 19817
I20260807 05:10:48.864660 8257 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.16.84:36549
Aug 07, 2026 5:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 5:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 5:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 5:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 5:10:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 5:10:50 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 05:10:50 dist-test-slave-49h3 krb5kdc[19389](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786079450, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.16.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:10:50.947873 8257 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.973s user 0.007s sys 0.202s
I20260807 05:10:50.948025 8257 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 05:10:50.948151 8257 mini_ranger_kms.cc:401] 127.8.16.84:36549/kms/v1/keys
Aug 07 05:10:51 dist-test-slave-49h3 krb5kdc[19389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079450, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.16.84@KRBTEST.COM for HTTP/127.8.16.84@KRBTEST.COM
Aug 07 05:10:51 dist-test-slave-49h3 krb5kdc[19389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079450, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.16.84@KRBTEST.COM for HTTP/127.8.16.84@KRBTEST.COM
Aug 07 05:10:53 dist-test-slave-49h3 krb5kdc[19389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:10:53.997815 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:36923
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:33507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:36923
--ranger_config_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.16.84:36549/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:10:54.109827 19870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:10:54.110132 19870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:10:54.110193 19870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:10:54.114362 19870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:10:54.114426 19870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:10:54.114452 19870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:10:54.114475 19870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:10:54.114502 19870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:10:54.120090 19870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:33507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.16.84:36549/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:36923
--ranger_config_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:36923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:10:54.121193 19870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:10:54.122244 19870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:10:54.130792 19876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:54.130826 19875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:54.130826 19878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:10:54.131209 19870 server_base.cc:1034] running on GCE node
I20260807 05:10:54.131765 19870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:10:54.133155 19870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:10:54.134346 19870 hybrid_clock.cc:648] HybridClock initialized: now 1786079454134314 us; error 42 us; skew 500 ppm
Aug 07 05:10:54 dist-test-slave-49h3 krb5kdc[19389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079454, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:10:54.142189 19870 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:10:54.143352 19870 webserver.cc:460] Webserver started at http://127.8.16.126:44909/ using document root <none> and password file <none>
I20260807 05:10:54.144006 19870 fs_manager.cc:359] Metadata directory not provided
I20260807 05:10:54.144088 19870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:10:54.144327 19870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 05:10:54 dist-test-slave-49h3 krb5kdc[19389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079454, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for HTTP/127.8.16.84@KRBTEST.COM
I20260807 05:10:54.200667 19870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "47cb8285ae364e1e9f926b94d8f0427d"
format_stamp: "Formatted at 2026-08-07 05:10:54 on dist-test-slave-49h3"
server_key: "baf6f9e744bf300a70a9fc4fddc47402"
server_key_iv: "2b3754b61f7ed31f2b2fe0f4b0641747"
server_key_version: "kuduclusterkey@0"
I20260807 05:10:54.201489 19870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "47cb8285ae364e1e9f926b94d8f0427d"
format_stamp: "Formatted at 2026-08-07 05:10:54 on dist-test-slave-49h3"
server_key: "baf6f9e744bf300a70a9fc4fddc47402"
server_key_iv: "2b3754b61f7ed31f2b2fe0f4b0641747"
server_key_version: "kuduclusterkey@0"
I20260807 05:10:54.206429 19870 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260807 05:10:54.224632 19886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:54.226339 19870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:10:54.226498 19870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "47cb8285ae364e1e9f926b94d8f0427d"
format_stamp: "Formatted at 2026-08-07 05:10:54 on dist-test-slave-49h3"
server_key: "baf6f9e744bf300a70a9fc4fddc47402"
server_key_iv: "2b3754b61f7ed31f2b2fe0f4b0641747"
server_key_version: "kuduclusterkey@0"
I20260807 05:10:54.226647 19870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:10:54.239346 19870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:10:54.242404 19870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:10:54.242595 19870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:10:54.250998 19870 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:36923
I20260807 05:10:54.251076 19939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:36923 every 8 connection(s)
I20260807 05:10:54.252314 19870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:10:54.255215 19940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:10:54.256022 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 19870
I20260807 05:10:54.256167 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:10:54.256481 8257 external_mini_cluster.cc:1428] Setting key 90dcd3cd6e951a205a83d665f7ee5e28
I20260807 05:10:54.261296 19940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d: Bootstrap starting.
I20260807 05:10:54.264156 19940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d: Neither blocks nor log segments found. Creating new log.
I20260807 05:10:54.265065 19940 log.cc:824] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d: Log is configured to *not* fsync() on all Append() calls
I20260807 05:10:54.267237 19940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d: No bootstrap required, opened a new log
Aug 07 05:10:54 dist-test-slave-49h3 krb5kdc[19389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:10:54.271659 19940 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47cb8285ae364e1e9f926b94d8f0427d" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36923 } }
I20260807 05:10:54.271945 19940 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:10:54.272027 19940 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47cb8285ae364e1e9f926b94d8f0427d, State: Initialized, Role: FOLLOWER
I20260807 05:10:54.272559 19940 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47cb8285ae364e1e9f926b94d8f0427d" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36923 } }
I20260807 05:10:54.272713 19940 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:10:54.272797 19940 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:10:54.272899 19940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:10:54.274060 19940 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47cb8285ae364e1e9f926b94d8f0427d" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36923 } }
I20260807 05:10:54.274416 19940 leader_election.cc:302] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47cb8285ae364e1e9f926b94d8f0427d; no voters:
I20260807 05:10:54.274744 19940 leader_election.cc:288] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:10:54.275548 19946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:10:54.275802 19946 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d [term 1 LEADER]: Becoming Leader. State: Replica: 47cb8285ae364e1e9f926b94d8f0427d, State: Running, Role: LEADER
I20260807 05:10:54.276173 19946 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47cb8285ae364e1e9f926b94d8f0427d" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36923 } }
I20260807 05:10:54.276244 19940 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:10:54.276428 19943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:54.257990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36092 (local address 127.8.16.126:36923)
0807 05:10:54.261713 (+ 3723us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:54.261734 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:54.261871 (+ 137us) server_negotiation.cc:407] Connection header received
0807 05:10:54.262829 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:54.262854 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:54.263300 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:54.263672 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:54.264598 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:54.265910 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:54.267162 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:54.267834 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:54.267993 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:54.271133 (+ 3140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:10:54.271140 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:10:54.271160 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:10:54.271195 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:10:54.273138 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:10:54.273674 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:10:54.273680 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:10:54.273687 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:10:54.273819 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:10:54.274134 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:10:54.274137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:10:54.274139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:10:54.274640 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:10:54.274836 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:54.275137 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 05:10:54.275428 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3547,"thread_start_us":141,"threads_started":1}
I20260807 05:10:54.277978 19948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47cb8285ae364e1e9f926b94d8f0427d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47cb8285ae364e1e9f926b94d8f0427d" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36923 } } }
I20260807 05:10:54.278095 19948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d [sys.catalog]: This master's current role is: LEADER
I20260807 05:10:54.278867 19940 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 05:10:54.279270 19947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47cb8285ae364e1e9f926b94d8f0427d. Latest consensus state: current_term: 1 leader_uuid: "47cb8285ae364e1e9f926b94d8f0427d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47cb8285ae364e1e9f926b94d8f0427d" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36923 } } }
I20260807 05:10:54.279369 19947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d [sys.catalog]: This master's current role is: LEADER
I20260807 05:10:54.280009 19957 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:10:54.287104 19957 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:10:54.293850 19957 catalog_manager.cc:1347] Generated new cluster ID: f2fe9b346dc64109806781b8cf2db237
I20260807 05:10:54.293928 19957 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:10:54.323324 19957 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:10:54.324745 19957 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:10:54.341049 19957 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d: Generated new TSK 0
I20260807 05:10:54.342068 19957 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
I20260807 05:10:55.036599 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.16.84:36549/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36923
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:33507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:10:55.173009 19987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:10:55.173267 19987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:10:55.173329 19987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:10:55.177560 19987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:10:55.177675 19987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:10:55.177783 19987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:10:55.184855 19987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:33507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.16.84:36549/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:10:55.186491 19987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:10:55.187745 19987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:10:55.197402 19995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:55.197570 19994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:55.197402 19997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:10:55.198742 19987 server_base.cc:1034] running on GCE node
I20260807 05:10:55.199249 19987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:10:55.200115 19987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:10:55.201339 19987 hybrid_clock.cc:648] HybridClock initialized: now 1786079455201316 us; error 35 us; skew 500 ppm
Aug 07 05:10:55 dist-test-slave-49h3 krb5kdc[19389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079455, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:10:55.204931 19987 init.cc:377] Logged in from keytab as kudu/127.8.16.65@KRBTEST.COM (short username kudu)
I20260807 05:10:55.206138 19987 webserver.cc:460] Webserver started at http://127.8.16.65:37101/ using document root <none> and password file <none>
I20260807 05:10:55.206853 19987 fs_manager.cc:359] Metadata directory not provided
I20260807 05:10:55.206957 19987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:10:55.207216 19987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 05:10:55 dist-test-slave-49h3 krb5kdc[19389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079455, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for HTTP/127.8.16.84@KRBTEST.COM
I20260807 05:10:55.222780 19987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "9e638ceb61704be3b8d62e4fce768536"
format_stamp: "Formatted at 2026-08-07 05:10:55 on dist-test-slave-49h3"
server_key: "154598cc85fee78ad68d20ad48eaa3ae"
server_key_iv: "b9a87b510a948c032a65030b687040fc"
server_key_version: "kuduclusterkey@0"
I20260807 05:10:55.223750 19987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "9e638ceb61704be3b8d62e4fce768536"
format_stamp: "Formatted at 2026-08-07 05:10:55 on dist-test-slave-49h3"
server_key: "154598cc85fee78ad68d20ad48eaa3ae"
server_key_iv: "b9a87b510a948c032a65030b687040fc"
server_key_version: "kuduclusterkey@0"
I20260807 05:10:55.230635 19987 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.000s
I20260807 05:10:55.246312 20004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:55.247982 19987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 05:10:55.248198 19987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "9e638ceb61704be3b8d62e4fce768536"
format_stamp: "Formatted at 2026-08-07 05:10:55 on dist-test-slave-49h3"
server_key: "154598cc85fee78ad68d20ad48eaa3ae"
server_key_iv: "b9a87b510a948c032a65030b687040fc"
server_key_version: "kuduclusterkey@0"
I20260807 05:10:55.248420 19987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:10:55.287221 19987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:10:55.290977 19987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:10:55.291244 19987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:10:55.292023 19987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:10:55.293148 19987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:10:55.293259 19987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:55.293390 19987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:10:55.293490 19987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:55.304306 19987 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:45623
I20260807 05:10:55.304484 20126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:45623 every 8 connection(s)
I20260807 05:10:55.305656 19987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:10:55.314673 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 19987
I20260807 05:10:55.314821 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:10:55.315145 8257 external_mini_cluster.cc:1428] Setting key 3f6fb2e6afd4cda0fca70a8762c08984
Aug 07 05:10:55 dist-test-slave-49h3 krb5kdc[19389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079455, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:10:55.328382 20129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:55.308293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36923 (local address 127.8.16.65:58869)
0807 05:10:55.313550 (+ 5257us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:55.313620 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:55.314743 (+ 1123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:55.315802 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:55.315812 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:55.316192 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:55.316882 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:55.316902 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:55.317980 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:55.317983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:55.319255 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:55.319267 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:55.320012 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:55.320016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:55.320225 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:55.320952 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:55.320974 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:55.322701 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:10:55.324916 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:10:55.324923 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:10:55.324939 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:10:55.325269 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:10:55.325585 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:10:55.325589 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:10:55.325591 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:10:55.325756 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:10:55.326254 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:10:55.326261 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:10:55.326683 (+ 422us) client_negotiation.cc:769] Sending connection context
0807 05:10:55.326896 (+ 213us) client_negotiation.cc:240] Negotiation successful
0807 05:10:55.327264 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4969,"thread_start_us":144,"threads_started":1}
I20260807 05:10:55.329268 20130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:55.309483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:58869 (local address 127.8.16.126:36923)
0807 05:10:55.315244 (+ 5761us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:55.315254 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:55.315271 (+ 17us) server_negotiation.cc:407] Connection header received
0807 05:10:55.315317 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:55.315320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:55.315372 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:55.315459 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:55.317062 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:55.317828 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:55.319393 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:55.319738 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:55.319831 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:55.322898 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:10:55.322902 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:10:55.322905 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:10:55.322933 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:10:55.324788 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:10:55.325375 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:10:55.325378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:10:55.325380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:10:55.325492 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:10:55.325849 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:10:55.325852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:10:55.325854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:10:55.326104 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:10:55.326198 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:55.328969 (+ 2771us) server_negotiation.cc:299] Negotiation successful
0807 05:10:55.329111 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5666,"thread_start_us":112,"threads_started":1}
I20260807 05:10:55.330461 20127 heartbeater.cc:347] Connected to a master server at 127.8.16.126:36923
I20260807 05:10:55.330704 20127 heartbeater.cc:464] Registering TS with master...
I20260807 05:10:55.331354 20127 heartbeater.cc:511] Master 127.8.16.126:36923 requested a full tablet report, sending...
I20260807 05:10:55.333242 19898 ts_manager.cc:194] Registered new tserver with Master: 9e638ceb61704be3b8d62e4fce768536 (127.8.16.65:45623)
I20260807 05:10:55.334903 19898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:58869
WARNING: no policy specified for kudu/127.8.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
I20260807 05:10:55.365116 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.16.84:36549/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36923
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:33507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 05:10:55 dist-test-slave-49h3 krb5kdc[19389](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786079455, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 05:10:55.512681 20133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:10:55.513113 20133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:10:55.513190 20133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:10:55.519404 20133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:10:55.519556 20133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:10:55.519680 20133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:10:55.527778 20133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:33507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.16.84:36549/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:10:55.529367 20133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:10:55.530825 20133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:10:55.543279 20140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:10:55.543848 20133 server_base.cc:1034] running on GCE node
W20260807 05:10:55.543527 20142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:55.543623 20139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:10:55.544616 20133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:10:55.545425 20133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:10:55.546633 20133 hybrid_clock.cc:648] HybridClock initialized: now 1786079455546608 us; error 38 us; skew 500 ppm
Aug 07 05:10:55 dist-test-slave-49h3 krb5kdc[19389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079455, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:10:55.550271 20133 init.cc:377] Logged in from keytab as kudu/127.8.16.66@KRBTEST.COM (short username kudu)
I20260807 05:10:55.551446 20133 webserver.cc:460] Webserver started at http://127.8.16.66:39453/ using document root <none> and password file <none>
I20260807 05:10:55.552158 20133 fs_manager.cc:359] Metadata directory not provided
I20260807 05:10:55.552245 20133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:10:55.552562 20133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 05:10:55 dist-test-slave-49h3 krb5kdc[19389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079455, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for HTTP/127.8.16.84@KRBTEST.COM
I20260807 05:10:55.577494 20133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "6d3c7284308a48edaa812d63592d3962"
format_stamp: "Formatted at 2026-08-07 05:10:55 on dist-test-slave-49h3"
server_key: "7e75b6850683902f05f7b5333f9ca973"
server_key_iv: "4ac7d592f59f4cad4d9be815646106f8"
server_key_version: "kuduclusterkey@0"
I20260807 05:10:55.578604 20133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "6d3c7284308a48edaa812d63592d3962"
format_stamp: "Formatted at 2026-08-07 05:10:55 on dist-test-slave-49h3"
server_key: "7e75b6850683902f05f7b5333f9ca973"
server_key_iv: "4ac7d592f59f4cad4d9be815646106f8"
server_key_version: "kuduclusterkey@0"
I20260807 05:10:55.584113 20133 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260807 05:10:55.606204 20150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:55.607883 20133 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 05:10:55.608068 20133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "6d3c7284308a48edaa812d63592d3962"
format_stamp: "Formatted at 2026-08-07 05:10:55 on dist-test-slave-49h3"
server_key: "7e75b6850683902f05f7b5333f9ca973"
server_key_iv: "4ac7d592f59f4cad4d9be815646106f8"
server_key_version: "kuduclusterkey@0"
I20260807 05:10:55.608263 20133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:10:55.626943 20133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:10:55.630599 20133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:10:55.630798 20133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:10:55.631435 20133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:10:55.632449 20133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:10:55.632548 20133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:55.632618 20133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:10:55.632658 20133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:55.643153 20133 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:33509
I20260807 05:10:55.643225 20269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:33509 every 8 connection(s)
I20260807 05:10:55.644205 20133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:10:55.647097 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 20133
I20260807 05:10:55.647224 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:10:55.647535 8257 external_mini_cluster.cc:1428] Setting key 545f9caf2ca9ba052fdd9f1915b68359
Aug 07 05:10:55 dist-test-slave-49h3 krb5kdc[19389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079455, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:10:55.658308 20130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:55.646100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:37107 (local address 127.8.16.126:36923)
0807 05:10:55.646252 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:55.646256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:55.647091 (+ 835us) server_negotiation.cc:407] Connection header received
0807 05:10:55.648111 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:55.648115 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:55.648171 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:55.648272 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:55.649500 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:55.650019 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:55.651152 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:55.651483 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:55.651554 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:55.654239 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:10:55.654243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:10:55.654245 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:10:55.654277 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:10:55.655827 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:10:55.656438 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:10:55.656442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:10:55.656444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:10:55.656537 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:10:55.656937 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:10:55.656941 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:10:55.656943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:10:55.657189 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:10:55.657290 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:55.657974 (+ 684us) server_negotiation.cc:299] Negotiation successful
0807 05:10:55.658115 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
WARNING: no policy specified for kudu/127.8.16.67@KRBTEST.COM; defaulting to no policy
I20260807 05:10:55.659340 20272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:55.646492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36923 (local address 127.8.16.66:37107)
0807 05:10:55.646944 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:55.646996 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:55.647846 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:55.648455 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:55.648463 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:55.648807 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:55.649352 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:55.649366 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:55.650138 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:55.650142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:55.651007 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:55.651018 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:55.651765 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:55.651769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:55.651968 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:55.652611 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:55.652627 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:55.654076 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:10:55.655957 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:10:55.655964 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:10:55.655981 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:10:55.656329 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:10:55.656630 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:10:55.656633 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:10:55.656635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:10:55.656839 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:10:55.657303 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:10:55.657310 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:10:55.657686 (+ 376us) client_negotiation.cc:769] Sending connection context
0807 05:10:55.657911 (+ 225us) client_negotiation.cc:240] Negotiation successful
0807 05:10:55.658257 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":107,"threads_started":1}
I20260807 05:10:55.660493 20270 heartbeater.cc:347] Connected to a master server at 127.8.16.126:36923
I20260807 05:10:55.660737 20270 heartbeater.cc:464] Registering TS with master...
I20260807 05:10:55.661306 20270 heartbeater.cc:511] Master 127.8.16.126:36923 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.67@KRBTEST.COM" created.
I20260807 05:10:55.662621 19898 ts_manager.cc:194] Registered new tserver with Master: 6d3c7284308a48edaa812d63592d3962 (127.8.16.66:33509)
I20260807 05:10:55.663394 19898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:37107
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
I20260807 05:10:55.685201 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.16.84:36549/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36923
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:33507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 05:10:55 dist-test-slave-49h3 krb5kdc[19389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079455, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for HTTP/127.8.16.84@KRBTEST.COM
W20260807 05:10:55.848515 20275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:10:55.848873 20275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:10:55.848929 20275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:10:55.854885 20275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:10:55.855039 20275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:10:55.855177 20275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:10:55.863097 20275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:33507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.16.84:36549/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:10:55.864698 20275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:10:55.866091 20275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:10:55.877914 20283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:55.877919 20280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:10:55.878423 20281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:10:55.879290 20275 server_base.cc:1034] running on GCE node
I20260807 05:10:55.879763 20275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:10:55.880615 20275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:10:55.881798 20275 hybrid_clock.cc:648] HybridClock initialized: now 1786079455881771 us; error 43 us; skew 500 ppm
Aug 07 05:10:55 dist-test-slave-49h3 krb5kdc[19389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079455, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:10:55.885696 20275 init.cc:377] Logged in from keytab as kudu/127.8.16.67@KRBTEST.COM (short username kudu)
I20260807 05:10:55.886749 20275 webserver.cc:460] Webserver started at http://127.8.16.67:35799/ using document root <none> and password file <none>
I20260807 05:10:55.887431 20275 fs_manager.cc:359] Metadata directory not provided
I20260807 05:10:55.887517 20275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:10:55.887741 20275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 05:10:55 dist-test-slave-49h3 krb5kdc[19389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079455, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for HTTP/127.8.16.84@KRBTEST.COM
I20260807 05:10:55.902525 20275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "4c31ab65f447456886fae9e083dc2446"
format_stamp: "Formatted at 2026-08-07 05:10:55 on dist-test-slave-49h3"
server_key: "bf9465a34aa40f854d5ac2874bb28153"
server_key_iv: "886508184ee429a5f0ab206017b3136e"
server_key_version: "kuduclusterkey@0"
I20260807 05:10:55.903242 20275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "4c31ab65f447456886fae9e083dc2446"
format_stamp: "Formatted at 2026-08-07 05:10:55 on dist-test-slave-49h3"
server_key: "bf9465a34aa40f854d5ac2874bb28153"
server_key_iv: "886508184ee429a5f0ab206017b3136e"
server_key_version: "kuduclusterkey@0"
I20260807 05:10:55.911430 20275 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.000s
I20260807 05:10:55.929167 20291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:55.931010 20275 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:10:55.931159 20275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "4c31ab65f447456886fae9e083dc2446"
format_stamp: "Formatted at 2026-08-07 05:10:55 on dist-test-slave-49h3"
server_key: "bf9465a34aa40f854d5ac2874bb28153"
server_key_iv: "886508184ee429a5f0ab206017b3136e"
server_key_version: "kuduclusterkey@0"
I20260807 05:10:55.931303 20275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:10:55.966782 20275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:10:55.969753 20275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:10:55.969976 20275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:10:55.970582 20275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:10:55.971434 20275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:10:55.971484 20275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:55.971537 20275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:10:55.971565 20275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:10:55.981840 20275 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:44651
I20260807 05:10:55.982021 20411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:44651 every 8 connection(s)
I20260807 05:10:55.982810 20275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:10:55.988824 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 20275
I20260807 05:10:55.988966 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:10:55.989284 8257 external_mini_cluster.cc:1428] Setting key 95be4f89608e25af6770e8ad6198ab79
Aug 07 05:10:55 dist-test-slave-49h3 krb5kdc[19389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079455, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:10:55.996914 20130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:55.984603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:58783 (local address 127.8.16.126:36923)
0807 05:10:55.984738 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:55.984742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:55.985686 (+ 944us) server_negotiation.cc:407] Connection header received
0807 05:10:55.986685 (+ 999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:55.986688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:55.986748 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:55.986821 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:55.987900 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:55.988550 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:55.990243 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:55.990542 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:55.990624 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:55.993384 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:10:55.993387 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:10:55.993389 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:10:55.993415 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:10:55.994914 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:10:55.995403 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:10:55.995407 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:10:55.995409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:10:55.995489 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:10:55.995814 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:10:55.995817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:10:55.995819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:10:55.996009 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:10:55.996108 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:55.996621 (+ 513us) server_negotiation.cc:299] Negotiation successful
0807 05:10:55.996745 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 05:10:55.998083 20415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:55.985038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36923 (local address 127.8.16.67:58783)
0807 05:10:55.985529 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:55.985582 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:55.986473 (+ 891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:55.987003 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:55.987009 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:55.987286 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:10:55.987751 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:55.987764 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:55.988728 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:55.988732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:55.990111 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:55.990122 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:55.990835 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:55.990839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:55.991033 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:55.991793 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:10:55.991815 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:10:55.993217 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:10:55.995022 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:10:55.995029 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:10:55.995048 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:10:55.995309 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:10:55.995568 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:10:55.995578 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:10:55.995580 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:10:55.995728 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:10:55.996098 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:10:55.996105 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:10:55.996382 (+ 277us) client_negotiation.cc:769] Sending connection context
0807 05:10:55.996559 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 05:10:55.997173 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":119,"threads_started":1}
I20260807 05:10:55.999274 20413 heartbeater.cc:347] Connected to a master server at 127.8.16.126:36923
I20260807 05:10:55.999507 20413 heartbeater.cc:464] Registering TS with master...
I20260807 05:10:55.999993 20413 heartbeater.cc:511] Master 127.8.16.126:36923 requested a full tablet report, sending...
I20260807 05:10:56.000952 19898 ts_manager.cc:194] Registered new tserver with Master: 4c31ab65f447456886fae9e083dc2446 (127.8.16.67:44651)
I20260807 05:10:56.001567 19898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:58783
I20260807 05:10:56.003803 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:10:56.013118 20130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:56.005914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48542 (local address 127.8.16.126:36923)
0807 05:10:56.006030 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:56.006034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:56.006307 (+ 273us) server_negotiation.cc:407] Connection header received
0807 05:10:56.006412 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:56.006415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:56.006459 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:56.006523 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:10:56.007226 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.007746 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.008569 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.008806 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.008891 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:56.010228 (+ 1337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:10:56.010232 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:10:56.010234 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:10:56.010262 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:10:56.011595 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:10:56.011989 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:10:56.011992 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:10:56.011994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:10:56.012069 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:10:56.012317 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:10:56.012320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:10:56.012322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:10:56.012490 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:10:56.012604 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:56.012834 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 05:10:56.012950 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 05:10:56.016253 19898 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48542:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:10:56.019531 19898 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:10:56.037079 20425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:56.029018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:44651 (local address 127.0.0.1:55056)
0807 05:10:56.030251 (+ 1233us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:56.030305 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:56.030446 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:56.031742 (+ 1296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:56.031745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:56.031761 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:10:56.031845 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:56.031852 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.033473 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.033476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:56.035086 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:56.035098 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.036398 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.036402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:56.036521 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:56.036562 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 05:10:56.036682 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 05:10:56.036913 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1066,"mutex_wait_us":59,"spinlock_wait_cycles":42112,"thread_start_us":91,"threads_started":1}
I20260807 05:10:56.037554 20426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:56.028351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52654 (local address 127.8.16.66:33509)
0807 05:10:56.030016 (+ 1665us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:56.030030 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:56.030389 (+ 359us) server_negotiation.cc:407] Connection header received
0807 05:10:56.030587 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:56.030593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:56.030872 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:56.031027 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:10:56.031518 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.033500 (+ 1982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.035952 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.036731 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.036806 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:56.036934 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:56.037269 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 05:10:56.037370 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1554,"thread_start_us":87,"threads_started":1}
I20260807 05:10:56.037990 20429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:56.029427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55056 (local address 127.8.16.67:44651)
0807 05:10:56.030733 (+ 1306us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:56.030744 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:56.030766 (+ 22us) server_negotiation.cc:407] Connection header received
0807 05:10:56.031244 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:56.031250 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:56.031472 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:56.031608 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:10:56.032006 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.033328 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.035265 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.036073 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.036182 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:56.036327 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:56.037749 (+ 1422us) server_negotiation.cc:299] Negotiation successful
0807 05:10:56.037845 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1209,"thread_start_us":92,"threads_started":1}
I20260807 05:10:56.038169 20427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:56.028229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:33509 (local address 127.0.0.1:52654)
0807 05:10:56.030251 (+ 2022us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:56.030305 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:56.030446 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:56.031233 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:56.031239 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:56.031263 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:10:56.031386 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:56.031393 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.033637 (+ 2244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.033640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:56.035800 (+ 2160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:56.035810 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.037026 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.037029 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:56.037112 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:56.037133 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 05:10:56.037942 (+ 809us) client_negotiation.cc:240] Negotiation successful
0807 05:10:56.038039 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1845,"mutex_wait_us":143,"thread_start_us":125,"threads_started":1}
I20260807 05:10:56.040890 20343 tablet_service.cc:1467] Processing CreateTablet for tablet 53eb4cc31b9f423a8293c179e00cd39a (DEFAULT_TABLE table=test-table [id=88f02443f5bc43268e2636a4316251a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:10:56.041069 20200 tablet_service.cc:1467] Processing CreateTablet for tablet 53eb4cc31b9f423a8293c179e00cd39a (DEFAULT_TABLE table=test-table [id=88f02443f5bc43268e2636a4316251a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:10:56.042016 20343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53eb4cc31b9f423a8293c179e00cd39a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:10:56.042115 20200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53eb4cc31b9f423a8293c179e00cd39a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:10:56.045986 20430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:56.029018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:45623 (local address 127.0.0.1:52334)
0807 05:10:56.030911 (+ 1893us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:56.030924 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:56.030995 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:56.036332 (+ 5337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:56.036335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:56.036352 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:10:56.036451 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:56.036457 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.042447 (+ 5990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.042451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:56.044373 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:56.044384 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.045554 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.045558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:56.045678 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:56.045706 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 05:10:56.045761 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 05:10:56.045838 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1835,"mutex_wait_us":678,"thread_start_us":81,"threads_started":1}
I20260807 05:10:56.046911 20428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:56.029207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52334 (local address 127.8.16.65:45623)
0807 05:10:56.035712 (+ 6505us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:56.035718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:56.035756 (+ 38us) server_negotiation.cc:407] Connection header received
0807 05:10:56.035834 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:56.035839 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:56.036024 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:56.036181 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:10:56.041163 (+ 4982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.042314 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.044524 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.045365 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.046456 (+ 1091us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:56.046606 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:56.046674 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 05:10:56.046769 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6389,"thread_start_us":72,"threads_started":1}
I20260807 05:10:56.050030 20055 tablet_service.cc:1467] Processing CreateTablet for tablet 53eb4cc31b9f423a8293c179e00cd39a (DEFAULT_TABLE table=test-table [id=88f02443f5bc43268e2636a4316251a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:10:56.050559 20431 tablet_bootstrap.cc:492] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446: Bootstrap starting.
I20260807 05:10:56.051086 20055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53eb4cc31b9f423a8293c179e00cd39a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:10:56.054658 20431 tablet_bootstrap.cc:654] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446: Neither blocks nor log segments found. Creating new log.
I20260807 05:10:56.055090 20432 tablet_bootstrap.cc:492] T 53eb4cc31b9f423a8293c179e00cd39a P 6d3c7284308a48edaa812d63592d3962: Bootstrap starting.
I20260807 05:10:56.058279 20432 tablet_bootstrap.cc:654] T 53eb4cc31b9f423a8293c179e00cd39a P 6d3c7284308a48edaa812d63592d3962: Neither blocks nor log segments found. Creating new log.
I20260807 05:10:56.058976 20431 log.cc:824] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446: Log is configured to *not* fsync() on all Append() calls
I20260807 05:10:56.060181 20432 log.cc:824] T 53eb4cc31b9f423a8293c179e00cd39a P 6d3c7284308a48edaa812d63592d3962: Log is configured to *not* fsync() on all Append() calls
I20260807 05:10:56.061539 20433 tablet_bootstrap.cc:492] T 53eb4cc31b9f423a8293c179e00cd39a P 9e638ceb61704be3b8d62e4fce768536: Bootstrap starting.
I20260807 05:10:56.062427 20431 tablet_bootstrap.cc:492] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446: No bootstrap required, opened a new log
I20260807 05:10:56.062685 20431 ts_tablet_manager.cc:1397] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260807 05:10:56.063771 20433 tablet_bootstrap.cc:654] T 53eb4cc31b9f423a8293c179e00cd39a P 9e638ceb61704be3b8d62e4fce768536: Neither blocks nor log segments found. Creating new log.
I20260807 05:10:56.064625 20433 log.cc:824] T 53eb4cc31b9f423a8293c179e00cd39a P 9e638ceb61704be3b8d62e4fce768536: Log is configured to *not* fsync() on all Append() calls
I20260807 05:10:56.066881 20433 tablet_bootstrap.cc:492] T 53eb4cc31b9f423a8293c179e00cd39a P 9e638ceb61704be3b8d62e4fce768536: No bootstrap required, opened a new log
I20260807 05:10:56.067072 20433 ts_tablet_manager.cc:1397] T 53eb4cc31b9f423a8293c179e00cd39a P 9e638ceb61704be3b8d62e4fce768536: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260807 05:10:56.066918 20431 raft_consensus.cc:348] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d3c7284308a48edaa812d63592d3962" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33509 } } peers { permanent_uuid: "9e638ceb61704be3b8d62e4fce768536" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45623 } } peers { permanent_uuid: "4c31ab65f447456886fae9e083dc2446" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 44651 } }
I20260807 05:10:56.067204 20431 raft_consensus.cc:374] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:10:56.067253 20431 raft_consensus.cc:729] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c31ab65f447456886fae9e083dc2446, State: Initialized, Role: FOLLOWER
I20260807 05:10:56.067776 20431 consensus_queue.cc:260] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d3c7284308a48edaa812d63592d3962" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33509 } } peers { permanent_uuid: "9e638ceb61704be3b8d62e4fce768536" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45623 } } peers { permanent_uuid: "4c31ab65f447456886fae9e083dc2446" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 44651 } }
I20260807 05:10:56.068771 20431 ts_tablet_manager.cc:1428] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 05:10:56.069154 20413 heartbeater.cc:503] Master 127.8.16.126:36923 was elected leader, sending a full tablet report...
I20260807 05:10:56.071149 20433 raft_consensus.cc:348] T 53eb4cc31b9f423a8293c179e00cd39a P 9e638ceb61704be3b8d62e4fce768536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d3c7284308a48edaa812d63592d3962" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33509 } } peers { permanent_uuid: "9e638ceb61704be3b8d62e4fce768536" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45623 } } peers { permanent_uuid: "4c31ab65f447456886fae9e083dc2446" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 44651 } }
I20260807 05:10:56.071388 20433 raft_consensus.cc:374] T 53eb4cc31b9f423a8293c179e00cd39a P 9e638ceb61704be3b8d62e4fce768536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:10:56.071439 20433 raft_consensus.cc:729] T 53eb4cc31b9f423a8293c179e00cd39a P 9e638ceb61704be3b8d62e4fce768536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e638ceb61704be3b8d62e4fce768536, State: Initialized, Role: FOLLOWER
I20260807 05:10:56.073150 20433 consensus_queue.cc:260] T 53eb4cc31b9f423a8293c179e00cd39a P 9e638ceb61704be3b8d62e4fce768536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d3c7284308a48edaa812d63592d3962" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33509 } } peers { permanent_uuid: "9e638ceb61704be3b8d62e4fce768536" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45623 } } peers { permanent_uuid: "4c31ab65f447456886fae9e083dc2446" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 44651 } }
I20260807 05:10:56.074285 20433 ts_tablet_manager.cc:1428] T 53eb4cc31b9f423a8293c179e00cd39a P 9e638ceb61704be3b8d62e4fce768536: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260807 05:10:56.074707 20432 tablet_bootstrap.cc:492] T 53eb4cc31b9f423a8293c179e00cd39a P 6d3c7284308a48edaa812d63592d3962: No bootstrap required, opened a new log
I20260807 05:10:56.074695 20127 heartbeater.cc:503] Master 127.8.16.126:36923 was elected leader, sending a full tablet report...
I20260807 05:10:56.074970 20432 ts_tablet_manager.cc:1397] T 53eb4cc31b9f423a8293c179e00cd39a P 6d3c7284308a48edaa812d63592d3962: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.000s
I20260807 05:10:56.078848 20432 raft_consensus.cc:348] T 53eb4cc31b9f423a8293c179e00cd39a P 6d3c7284308a48edaa812d63592d3962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d3c7284308a48edaa812d63592d3962" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33509 } } peers { permanent_uuid: "9e638ceb61704be3b8d62e4fce768536" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45623 } } peers { permanent_uuid: "4c31ab65f447456886fae9e083dc2446" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 44651 } }
I20260807 05:10:56.079071 20432 raft_consensus.cc:374] T 53eb4cc31b9f423a8293c179e00cd39a P 6d3c7284308a48edaa812d63592d3962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:10:56.079172 20432 raft_consensus.cc:729] T 53eb4cc31b9f423a8293c179e00cd39a P 6d3c7284308a48edaa812d63592d3962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d3c7284308a48edaa812d63592d3962, State: Initialized, Role: FOLLOWER
I20260807 05:10:56.079584 20432 consensus_queue.cc:260] T 53eb4cc31b9f423a8293c179e00cd39a P 6d3c7284308a48edaa812d63592d3962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d3c7284308a48edaa812d63592d3962" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33509 } } peers { permanent_uuid: "9e638ceb61704be3b8d62e4fce768536" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45623 } } peers { permanent_uuid: "4c31ab65f447456886fae9e083dc2446" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 44651 } }
I20260807 05:10:56.080197 20270 heartbeater.cc:503] Master 127.8.16.126:36923 was elected leader, sending a full tablet report...
I20260807 05:10:56.080564 20432 ts_tablet_manager.cc:1428] T 53eb4cc31b9f423a8293c179e00cd39a P 6d3c7284308a48edaa812d63592d3962: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
Aug 07 05:10:56 dist-test-slave-49h3 krb5kdc[19389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079455, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for HTTP/127.8.16.84@KRBTEST.COM
W20260807 05:10:56.145663 20271 tablet.cc:2369] T 53eb4cc31b9f423a8293c179e00cd39a P 6d3c7284308a48edaa812d63592d3962: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:10:56.186033 20437 raft_consensus.cc:482] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:10:56.186290 20437 raft_consensus.cc:504] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d3c7284308a48edaa812d63592d3962" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33509 } } peers { permanent_uuid: "9e638ceb61704be3b8d62e4fce768536" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45623 } } peers { permanent_uuid: "4c31ab65f447456886fae9e083dc2446" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 44651 } }
I20260807 05:10:56.187646 20437 leader_election.cc:288] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d3c7284308a48edaa812d63592d3962 (127.8.16.66:33509), 9e638ceb61704be3b8d62e4fce768536 (127.8.16.65:45623)
I20260807 05:10:56.193217 20415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:56.187884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:33509 (local address 127.8.16.67:45715)
0807 05:10:56.188084 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:56.188101 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:56.188193 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:56.189260 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:56.189263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:56.189284 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:10:56.189407 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:56.189414 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.190461 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.190464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:56.192161 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:56.192174 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.192815 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.192819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:56.192908 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:56.192931 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:10:56.193000 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 05:10:56.193076 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":424}
I20260807 05:10:56.193222 20426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:56.188703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:45715 (local address 127.8.16.66:33509)
0807 05:10:56.189018 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:56.189023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:56.189040 (+ 17us) server_negotiation.cc:407] Connection header received
0807 05:10:56.189095 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:56.189098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:56.189147 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:56.189237 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:10:56.189570 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.190327 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.192281 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.192699 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.192785 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:56.192847 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:56.193038 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 05:10:56.193125 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230}
I20260807 05:10:56.193320 20428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:56.188875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:45249 (local address 127.8.16.65:45623)
0807 05:10:56.189415 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:56.189419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:56.189433 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:10:56.189519 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:56.189522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:56.189571 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:56.189644 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:10:56.190055 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.190816 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.192323 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.192726 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.192828 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:56.192891 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:56.193106 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 05:10:56.193188 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456}
I20260807 05:10:56.194031 20221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53eb4cc31b9f423a8293c179e00cd39a" candidate_uuid: "4c31ab65f447456886fae9e083dc2446" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d3c7284308a48edaa812d63592d3962" is_pre_election: true
I20260807 05:10:56.194093 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:56.188164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:45623 (local address 127.8.16.67:45249)
0807 05:10:56.188518 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:56.188530 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:56.188621 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:56.189771 (+ 1150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:10:56.189774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:10:56.189793 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:10:56.189917 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:56.189924 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.190963 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.190967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:56.192157 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:10:56.192169 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.192847 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.192851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:10:56.192958 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:56.192981 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:10:56.193851 (+ 870us) client_negotiation.cc:240] Negotiation successful
0807 05:10:56.193928 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":64,"thread_start_us":137,"threads_started":1}
I20260807 05:10:56.194350 20221 raft_consensus.cc:2454] T 53eb4cc31b9f423a8293c179e00cd39a P 6d3c7284308a48edaa812d63592d3962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c31ab65f447456886fae9e083dc2446 in term 0.
I20260807 05:10:56.194877 20293 leader_election.cc:302] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c31ab65f447456886fae9e083dc2446, 6d3c7284308a48edaa812d63592d3962; no voters:
I20260807 05:10:56.195104 20078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53eb4cc31b9f423a8293c179e00cd39a" candidate_uuid: "4c31ab65f447456886fae9e083dc2446" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e638ceb61704be3b8d62e4fce768536" is_pre_election: true
I20260807 05:10:56.195180 20437 raft_consensus.cc:2781] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:10:56.195276 20437 raft_consensus.cc:482] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:10:56.195330 20437 raft_consensus.cc:3037] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:10:56.195398 20078 raft_consensus.cc:2454] T 53eb4cc31b9f423a8293c179e00cd39a P 9e638ceb61704be3b8d62e4fce768536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c31ab65f447456886fae9e083dc2446 in term 0.
I20260807 05:10:56.196802 20437 raft_consensus.cc:504] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d3c7284308a48edaa812d63592d3962" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33509 } } peers { permanent_uuid: "9e638ceb61704be3b8d62e4fce768536" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45623 } } peers { permanent_uuid: "4c31ab65f447456886fae9e083dc2446" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 44651 } }
I20260807 05:10:56.197178 20437 leader_election.cc:288] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446 [CANDIDATE]: Term 1 election: Requested vote from peers 6d3c7284308a48edaa812d63592d3962 (127.8.16.66:33509), 9e638ceb61704be3b8d62e4fce768536 (127.8.16.65:45623)
I20260807 05:10:56.197619 20221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53eb4cc31b9f423a8293c179e00cd39a" candidate_uuid: "4c31ab65f447456886fae9e083dc2446" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d3c7284308a48edaa812d63592d3962"
I20260807 05:10:56.197654 20078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53eb4cc31b9f423a8293c179e00cd39a" candidate_uuid: "4c31ab65f447456886fae9e083dc2446" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e638ceb61704be3b8d62e4fce768536"
I20260807 05:10:56.197769 20078 raft_consensus.cc:3037] T 53eb4cc31b9f423a8293c179e00cd39a P 9e638ceb61704be3b8d62e4fce768536 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:10:56.197775 20221 raft_consensus.cc:3037] T 53eb4cc31b9f423a8293c179e00cd39a P 6d3c7284308a48edaa812d63592d3962 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:10:56.199015 20078 raft_consensus.cc:2454] T 53eb4cc31b9f423a8293c179e00cd39a P 9e638ceb61704be3b8d62e4fce768536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c31ab65f447456886fae9e083dc2446 in term 1.
I20260807 05:10:56.199074 20221 raft_consensus.cc:2454] T 53eb4cc31b9f423a8293c179e00cd39a P 6d3c7284308a48edaa812d63592d3962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c31ab65f447456886fae9e083dc2446 in term 1.
I20260807 05:10:56.199414 20293 leader_election.cc:302] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c31ab65f447456886fae9e083dc2446, 6d3c7284308a48edaa812d63592d3962; no voters:
I20260807 05:10:56.199689 20437 raft_consensus.cc:2781] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:10:56.199922 20437 raft_consensus.cc:686] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446 [term 1 LEADER]: Becoming Leader. State: Replica: 4c31ab65f447456886fae9e083dc2446, State: Running, Role: LEADER
I20260807 05:10:56.200284 20437 consensus_queue.cc:237] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d3c7284308a48edaa812d63592d3962" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33509 } } peers { permanent_uuid: "9e638ceb61704be3b8d62e4fce768536" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45623 } } peers { permanent_uuid: "4c31ab65f447456886fae9e083dc2446" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 44651 } }
I20260807 05:10:56.203635 19898 catalog_manager.cc:5495] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c31ab65f447456886fae9e083dc2446 (127.8.16.67). New cstate: current_term: 1 leader_uuid: "4c31ab65f447456886fae9e083dc2446" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d3c7284308a48edaa812d63592d3962" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e638ceb61704be3b8d62e4fce768536" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c31ab65f447456886fae9e083dc2446" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 44651 } health_report { overall_health: HEALTHY } } }
W20260807 05:10:56.234683 20414 tablet.cc:2369] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:10:56.249650 20429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:56.244314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55068 (local address 127.8.16.67:44651)
0807 05:10:56.244685 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0807 05:10:56.244689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:10:56.244699 (+ 10us) server_negotiation.cc:407] Connection header received
0807 05:10:56.244768 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:10:56.244770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:10:56.244823 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:10:56.244914 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:10:56.245210 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.246014 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.247302 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:10:56.247617 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:10:56.247736 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:10:56.248173 (+ 437us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:10:56.248771 (+ 598us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:10:56.248900 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 05:10:56.249343 (+ 443us) server_negotiation.cc:299] Negotiation successful
0807 05:10:56.249412 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296}
I20260807 05:10:56.261554 20078 raft_consensus.cc:1261] T 53eb4cc31b9f423a8293c179e00cd39a P 9e638ceb61704be3b8d62e4fce768536 [term 1 FOLLOWER]: Refusing update from remote peer 4c31ab65f447456886fae9e083dc2446: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:10:56.262403 20443 consensus_queue.cc:1035] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e638ceb61704be3b8d62e4fce768536" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:10:56.275234 20221 raft_consensus.cc:1261] T 53eb4cc31b9f423a8293c179e00cd39a P 6d3c7284308a48edaa812d63592d3962 [term 1 FOLLOWER]: Refusing update from remote peer 4c31ab65f447456886fae9e083dc2446: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:10:56.276141 20443 consensus_queue.cc:1035] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d3c7284308a48edaa812d63592d3962" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 33509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:10:56.283998 20447 mvcc.cc:205] Tried to move back new op lower bound from 7315781452838514688 to 7315781452599517184. Current Snapshot: MvccSnapshot[applied={T|T < 7315781452838514688}]
I20260807 05:10:56.294729 19898 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:10:56.294926 19898 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:10:56.297859 19898 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47cb8285ae364e1e9f926b94d8f0427d: Sending DeleteTablet for 3 replicas of tablet 53eb4cc31b9f423a8293c179e00cd39a
I20260807 05:10:56.298671 20200 tablet_service.cc:1514] Processing DeleteTablet for tablet 53eb4cc31b9f423a8293c179e00cd39a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:10:56 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:52654
I20260807 05:10:56.299880 20055 tablet_service.cc:1514] Processing DeleteTablet for tablet 53eb4cc31b9f423a8293c179e00cd39a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:10:56 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:52334
I20260807 05:10:56.301347 20343 tablet_service.cc:1514] Processing DeleteTablet for tablet 53eb4cc31b9f423a8293c179e00cd39a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:10:56 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:55056
I20260807 05:10:56.302883 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 19987
I20260807 05:10:56.308826 20456 tablet_replica.cc:330] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446: stopping tablet replica
I20260807 05:10:56.309248 20456 raft_consensus.cc:2229] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:10:56.310659 20456 raft_consensus.cc:2258] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:10:56.310973 20454 tablet_replica.cc:330] T 53eb4cc31b9f423a8293c179e00cd39a P 6d3c7284308a48edaa812d63592d3962: stopping tablet replica
I20260807 05:10:56.321149 20454 raft_consensus.cc:2229] T 53eb4cc31b9f423a8293c179e00cd39a P 6d3c7284308a48edaa812d63592d3962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:10:56.321785 20454 raft_consensus.cc:2258] T 53eb4cc31b9f423a8293c179e00cd39a P 6d3c7284308a48edaa812d63592d3962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:10:56.322249 20456 ts_tablet_manager.cc:1905] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:10:56.325182 20456 ts_tablet_manager.cc:1918] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260807 05:10:56.325785 19888 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.16.65:45623 (error 108)
I20260807 05:10:56.326776 20456 log.cc:1196] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/53eb4cc31b9f423a8293c179e00cd39a
I20260807 05:10:56.327327 20456 ts_tablet_manager.cc:1939] T 53eb4cc31b9f423a8293c179e00cd39a P 4c31ab65f447456886fae9e083dc2446: Deleting consensus metadata
I20260807 05:10:56.327584 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 20133
I20260807 05:10:56.346165 19889 catalog_manager.cc:4841] TS 4c31ab65f447456886fae9e083dc2446 (127.8.16.67:44651): tablet 53eb4cc31b9f423a8293c179e00cd39a (table test-table [id=88f02443f5bc43268e2636a4316251a0]) successfully deleted
I20260807 05:10:56.347326 20430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:10:56.347032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:45623 (local address 127.0.0.1:52336)
0807 05:10:56.347152 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:56.347230 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.65:45623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 05:10:56.347579 19888 catalog_manager.cc:4584] TS 9e638ceb61704be3b8d62e4fce768536 (127.8.16.65:45623): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 53eb4cc31b9f423a8293c179e00cd39a: Network error: Client connection negotiation failed: client connection to 127.8.16.65:45623: connect: Connection refused (error 111)
W20260807 05:10:56.348706 20430 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:10:56.348191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:33509 (local address 127.0.0.1:52664)
0807 05:10:56.348332 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 05:10:56.348345 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:10:56.348428 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:10:56.348616 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.66:33509: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 05:10:56.349392 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 20275
I20260807 05:10:56.357074 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 19870
I20260807 05:10:56.365887 8257 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 05:10:56.710736 8257 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 05:10:56.710909 8257 mini_ranger.cc:66] Stopping Ranger...
I20260807 05:10:57.055601 8257 mini_ranger.cc:68] Stopped Ranger
2026-08-07 05:10:57.055 UTC [19421] LOG: received smart shutdown request
2026-08-07 05:10:57.059 UTC [19421] LOG: background worker "logical replication launcher" (PID 19437) exited with exit code 1
2026-08-07 05:10:57.059 UTC [19427] LOG: shutting down
2026-08-07 05:10:57.209 UTC [19421] LOG: database system is shut down
2026-08-07T05:10:57Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (44280 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:10:57 dist-test-slave-49h3 krb5kdc[20467](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:10:57 dist-test-slave-49h3 krb5kdc[20467](info): set up 2 sockets
Aug 07 05:10:57 dist-test-slave-49h3 krb5kdc[20467](info): commencing operation
krb5kdc: starting...
W20260807 05:10:58.313117 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.979s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:10:58 dist-test-slave-49h3 krb5kdc[20467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:10:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:10:58Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/rangeradmin_127.8.16.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/rangeradmin_127.8.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/rangerlookup_127.8.16.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/rangerlookup_127.8.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/HTTP_127.8.16.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/HTTP_127.8.16.84@KRBTEST.COM.keytab.
I20260807 05:10:58.479245 8257 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskg_K4zd/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/postgres -l logfile start
2026-08-07 05:11:00.772 UTC [20499] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 05:11:00.772 UTC [20499] LOG: listening on IPv4 address "127.8.16.84", port 38983
2026-08-07 05:11:00.775 UTC [20499] LOG: listening on Unix socket "/tmp/.s.PGSQL.38983"
2026-08-07 05:11:00.802 UTC [20502] LOG: database system was shut down at 2026-08-07 05:10:59 UTC
2026-08-07 05:11:00.806 UTC [20499] LOG: database system is ready to accept connections
I20260807 05:11:01.693529 8257 mini_postgres.cc:96] Postgres bound to 38983
2026-08-07 05:11:01.700 UTC [20510] FATAL: database "slave" does not exist
127.8.16.84:38983 - accepting connections
I20260807 05:11:01.700989 8257 mini_ranger.cc:161] Starting Ranger...
I20260807 05:11:01.715718 8257 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 05:11:02.142366 8257 mini_ranger.cc:87] Created ranger Postgres database
I20260807 05:11:02.142437 8257 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ranger-admin
2026-08-07 05:11:02,408 [I] DB FLAVOR :POSTGRES
2026-08-07 05:11:02,408 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 05:11:02,408 [I] Checking connection..
2026-08-07 05:11:02,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 05:11:02,668 [I] Checking connection passed.
2026-08-07 05:11:02,669 [I] --------- Verifying version history table ---------
2026-08-07 05:11:02,669 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 05:11:02,928 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 05:11:02,928 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 05:11:03,181 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 05:11:03,181 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 05:11:03,182 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 05:11:03.419 UTC [20577] WARNING: there is no transaction in progress
2026-08-07 05:11:03,425 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 05:11:03,425 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 05:11:03,673 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 05:11:03,673 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 05:11:03,673 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 05:11:03,909 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 05:11:04,168 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-49h3.c.gcp-upstream.internal','N') ;"
2026-08-07 05:11:04,394 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 05:11:04,634 [I] Table x_portal_user does not exist in database ranger
2026-08-07 05:11:04,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 05:11:04,895 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 05:11:04,896 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 05:11:05,145 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 05:11:05,399 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 05:11:05,400 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 05:11:06.313 UTC [20689] WARNING: there is no transaction in progress
2026-08-07 05:11:06.329 UTC [20689] WARNING: there is no transaction in progress
2026-08-07 05:11:06.346 UTC [20689] WARNING: there is no transaction in progress
2026-08-07 05:11:06.362 UTC [20689] WARNING: there is no transaction in progress
2026-08-07 05:11:06.378 UTC [20689] WARNING: there is no transaction in progress
2026-08-07 05:11:06.394 UTC [20689] WARNING: there is no transaction in progress
2026-08-07 05:11:06.446 UTC [20689] WARNING: there is no transaction in progress
2026-08-07 05:11:06.452 UTC [20689] WARNING: there is no transaction in progress
2026-08-07 05:11:06.465 UTC [20689] WARNING: there is no transaction in progress
2026-08-07 05:11:06.475 UTC [20689] WARNING: there is no transaction in progress
2026-08-07 05:11:06.484 UTC [20689] WARNING: there is no transaction in progress
2026-08-07 05:11:06.498 UTC [20689] WARNING: there is no transaction in progress
2026-08-07 05:11:06.507 UTC [20689] WARNING: there is no transaction in progress
2026-08-07 05:11:06.513 UTC [20689] WARNING: there is no transaction in progress
2026-08-07 05:11:06.519 UTC [20689] WARNING: there is no transaction in progress
2026-08-07 05:11:06.526 UTC [20689] WARNING: there is no transaction in progress
2026-08-07 05:11:07,117 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 05:11:07,117 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 05:11:07,346 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 05:11:07,346 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 05:11:07,575 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 05:11:07,575 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 05:11:07,816 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 05:11:07,816 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 05:11:08,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 05:11:08,309 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-49h3.c.gcp-upstream.internal';"
2026-08-07 05:11:08,559 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 05:11:08,560 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 05:11:08,799 [I] DB_PATCHES have already been applied
I20260807 05:11:08.805341 8257 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ranger-admin:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskg_K4zd/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 05:11:08.805415 8257 mini_ranger.cc:193] Using host: 127.8.16.84
I20260807 05:11:08.809355 8257 mini_ranger.cc:227] Ranger admin URL: http://127.8.16.84:40981
Aug 07, 2026 5:11:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 5:11:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 5:11:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 5:11:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 5:11:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 5:11:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 5:11:09 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 05:11:09.818867 8257 mini_ranger.cc:160] Time spent starting Ranger: real 8.118s user 0.005s sys 0.005s
Aug 07, 2026 5:11:30 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 5:11:30 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 5:11:30 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 5:11:30 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 5:11:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 5:11:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 5:11:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 5:11:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 5:11:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 5:11:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 5:11:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 5:11:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 5:11:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 5:11:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 5:11:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 5:11:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 5:11:30 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 5:11:31 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 05:11:32.283353 8257 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/rangerkms_127.8.16.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/rangerkms_127.8.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.16.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.16.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.16.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/HTTP_127.8.16.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.16.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/HTTP_127.8.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 05:11:32 dist-test-slave-49h3 krb5kdc[20467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079492, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 05:11:32.361299 8257 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 05:11:32.371068 8257 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 05:11:33.112768 8257 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 05:11:33.112857 8257 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ranger-kms
I20260807 05:11:33.112874 8257 mini_ranger_kms.cc:227] Using postgres at 127.8.16.84:38983
2026-08-07 05:11:33,290 [I] DB FLAVOR :POSTGRES
2026-08-07 05:11:33,290 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 05:11:33,290 [I] Checking connection
2026-08-07 05:11:33,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:38983/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 05:11:33,527 [I] connection success
2026-08-07 05:11:33,528 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 05:11:33,528 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 05:11:33,528 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:38983/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 05:11:33,783 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 05:11:33,784 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 05:11:33,784 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 05:11:33,784 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.16.84:38983/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 05:11:34,051 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 05:11:34.519762 8257 mini_ranger_kms.cc:310] Created kms service
I20260807 05:11:34.694550 8257 mini_ranger_kms.cc:326] Created kudu user
I20260807 05:11:34.739866 8257 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 05:11:34 dist-test-slave-49h3 krb5kdc[20467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079492, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.16.84@KRBTEST.COM
I20260807 05:11:35.142302 8257 mini_ranger_kms.cc:384] Added ranger policy
I20260807 05:11:35.142525 8257 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ranger-kms:/tmp/dist-test-taskg_K4zd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskg_K4zd/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 05:11:35.142554 8257 mini_ranger_kms.cc:240] Using host: 127.8.16.84
I20260807 05:11:35.147147 8257 mini_ranger_kms.cc:276] Ranger KMS PID: 20884
I20260807 05:11:35.147233 8257 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.16.84:59173
Aug 07, 2026 5:11:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 5:11:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 5:11:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 5:11:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 5:11:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 5:11:36 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260807 05:11:37.267426 8257 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.906s user 0.013s sys 0.149s
I20260807 05:11:37.267563 8257 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 05:11:37.267658 8257 mini_ranger_kms.cc:401] 127.8.16.84:59173/kms/v1/keys
Aug 07 05:11:37 dist-test-slave-49h3 krb5kdc[20467](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786079497, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.16.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 05:11:37 dist-test-slave-49h3 krb5kdc[20467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079497, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.16.84@KRBTEST.COM for HTTP/127.8.16.84@KRBTEST.COM
Aug 07 05:11:38 dist-test-slave-49h3 krb5kdc[20467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079497, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.16.84@KRBTEST.COM for HTTP/127.8.16.84@KRBTEST.COM
Aug 07 05:11:40 dist-test-slave-49h3 krb5kdc[20467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:11:40.328548 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:34391
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:34391
--ranger_config_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.16.84:59173/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:11:40.453197 20936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:40.453511 20936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:40.453580 20936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:40.457614 20936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:11:40.457680 20936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:40.457706 20936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:40.457728 20936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:11:40.457756 20936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:11:40.463168 20936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.16.84:59173/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:34391
--ranger_config_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:34391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:40.464178 20936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:40.465204 20936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:11:40.473510 20942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:40.473635 20945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:40.473562 20943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:40.473541 20936 server_base.cc:1034] running on GCE node
I20260807 05:11:40.474308 20936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:40.475346 20936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:40.476505 20936 hybrid_clock.cc:648] HybridClock initialized: now 1786079500476488 us; error 29 us; skew 500 ppm
Aug 07 05:11:40 dist-test-slave-49h3 krb5kdc[20467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079500, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:40.479147 20936 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:11:40.479980 20936 webserver.cc:460] Webserver started at http://127.8.16.126:39809/ using document root <none> and password file <none>
I20260807 05:11:40.480477 20936 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:40.480535 20936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:11:40.480729 20936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 05:11:40 dist-test-slave-49h3 krb5kdc[20467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079500, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for HTTP/127.8.16.84@KRBTEST.COM
I20260807 05:11:40.538555 20936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "bc93185fc657494f926432313e6c2530"
format_stamp: "Formatted at 2026-08-07 05:11:40 on dist-test-slave-49h3"
server_key: "78117fa71823014ff3e9f0096be10398"
server_key_iv: "f38ee6516d1c8c21482de0ad6788e224"
server_key_version: "kuduclusterkey@0"
I20260807 05:11:40.539477 20936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "bc93185fc657494f926432313e6c2530"
format_stamp: "Formatted at 2026-08-07 05:11:40 on dist-test-slave-49h3"
server_key: "78117fa71823014ff3e9f0096be10398"
server_key_iv: "f38ee6516d1c8c21482de0ad6788e224"
server_key_version: "kuduclusterkey@0"
I20260807 05:11:40.544545 20936 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 05:11:40.563534 20952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:40.564886 20936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:11:40.565025 20936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "bc93185fc657494f926432313e6c2530"
format_stamp: "Formatted at 2026-08-07 05:11:40 on dist-test-slave-49h3"
server_key: "78117fa71823014ff3e9f0096be10398"
server_key_iv: "f38ee6516d1c8c21482de0ad6788e224"
server_key_version: "kuduclusterkey@0"
I20260807 05:11:40.565178 20936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:40.581734 20936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:40.584771 20936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:40.584973 20936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:40.592628 20936 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:34391
I20260807 05:11:40.592669 21005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:34391 every 8 connection(s)
I20260807 05:11:40.593699 20936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:11:40.597262 21006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:11:40.597294 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 20936
I20260807 05:11:40.597396 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:11:40.597715 8257 external_mini_cluster.cc:1428] Setting key 523b558d32092b65d9c3da2341cb29b2
I20260807 05:11:40.605134 21006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530: Bootstrap starting.
Aug 07 05:11:40 dist-test-slave-49h3 krb5kdc[20467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:11:40.607933 21006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530: Neither blocks nor log segments found. Creating new log.
I20260807 05:11:40.609316 21006 log.cc:824] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530: Log is configured to *not* fsync() on all Append() calls
I20260807 05:11:40.611572 21009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:40.599152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58426 (local address 127.8.16.126:34391)
0807 05:11:40.599563 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:40.599581 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:40.599618 (+ 37us) server_negotiation.cc:407] Connection header received
0807 05:11:40.600314 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:40.600333 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:40.600680 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:40.601014 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:40.601916 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:40.602953 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:40.603892 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:40.604391 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:40.604546 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:40.607120 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:40.607134 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:40.607146 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:40.607176 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:40.608683 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:40.609105 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:40.609110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:40.609114 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:40.609229 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:40.609541 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:40.609544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:40.609546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:40.610051 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:40.610248 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:40.610550 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 05:11:40.610844 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":114,"threads_started":1}
I20260807 05:11:40.612366 21006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530: No bootstrap required, opened a new log
I20260807 05:11:40.616482 21006 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc93185fc657494f926432313e6c2530" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 34391 } }
I20260807 05:11:40.616794 21006 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:11:40.616858 21006 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc93185fc657494f926432313e6c2530, State: Initialized, Role: FOLLOWER
I20260807 05:11:40.617329 21006 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc93185fc657494f926432313e6c2530" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 34391 } }
I20260807 05:11:40.617512 21006 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:11:40.617583 21006 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:11:40.617676 21006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:11:40.618856 21006 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc93185fc657494f926432313e6c2530" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 34391 } }
I20260807 05:11:40.619292 21006 leader_election.cc:302] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc93185fc657494f926432313e6c2530; no voters:
I20260807 05:11:40.619688 21006 leader_election.cc:288] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:11:40.619792 21011 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:11:40.620082 21011 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [term 1 LEADER]: Becoming Leader. State: Replica: bc93185fc657494f926432313e6c2530, State: Running, Role: LEADER
I20260807 05:11:40.620455 21011 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc93185fc657494f926432313e6c2530" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 34391 } }
I20260807 05:11:40.621021 21006 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:11:40.621578 21011 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc93185fc657494f926432313e6c2530. Latest consensus state: current_term: 1 leader_uuid: "bc93185fc657494f926432313e6c2530" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc93185fc657494f926432313e6c2530" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 34391 } } }
I20260807 05:11:40.621781 21011 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [sys.catalog]: This master's current role is: LEADER
I20260807 05:11:40.621978 21013 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc93185fc657494f926432313e6c2530" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc93185fc657494f926432313e6c2530" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 34391 } } }
I20260807 05:11:40.622116 21013 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [sys.catalog]: This master's current role is: LEADER
I20260807 05:11:40.622228 21019 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:11:40.625465 21019 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:11:40.625916 21006 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 05:11:40.634402 21019 catalog_manager.cc:1347] Generated new cluster ID: eb0da21a31d24952a1b4d1bd6b606bf5
I20260807 05:11:40.634495 21019 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:11:40.658102 21019 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:11:40.659538 21019 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:11:40.667280 21019 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530: Generated new TSK 0
I20260807 05:11:40.668346 21019 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
I20260807 05:11:41.390249 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.16.84:59173/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:34391
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:11:41.503240 21055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:41.503551 21055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:41.503675 21055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:41.507863 21055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:41.507923 21055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:41.507980 21055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:11:41.513275 21055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.16.84:59173/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:34391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:41.514353 21055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:41.515354 21055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:11:41.526100 21060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:41.526114 21061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:41.526247 21063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:41.526684 21055 server_base.cc:1034] running on GCE node
I20260807 05:11:41.527069 21055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:41.527805 21055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:41.531109 21055 hybrid_clock.cc:648] HybridClock initialized: now 1786079501531077 us; error 46 us; skew 500 ppm
Aug 07 05:11:41 dist-test-slave-49h3 krb5kdc[20467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079501, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:41.533833 21055 init.cc:377] Logged in from keytab as kudu/127.8.16.65@KRBTEST.COM (short username kudu)
I20260807 05:11:41.534706 21055 webserver.cc:460] Webserver started at http://127.8.16.65:35691/ using document root <none> and password file <none>
I20260807 05:11:41.535225 21055 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:41.535286 21055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:11:41.535470 21055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 05:11:41 dist-test-slave-49h3 krb5kdc[20467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079501, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for HTTP/127.8.16.84@KRBTEST.COM
I20260807 05:11:41.552088 21055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "e2835474566449cca400fb5945831651"
format_stamp: "Formatted at 2026-08-07 05:11:41 on dist-test-slave-49h3"
server_key: "e484965bbe4475ab86dcd178365d4982"
server_key_iv: "6f9be70792062950732591ef936d3fc9"
server_key_version: "kuduclusterkey@0"
I20260807 05:11:41.552935 21055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "e2835474566449cca400fb5945831651"
format_stamp: "Formatted at 2026-08-07 05:11:41 on dist-test-slave-49h3"
server_key: "e484965bbe4475ab86dcd178365d4982"
server_key_iv: "6f9be70792062950732591ef936d3fc9"
server_key_version: "kuduclusterkey@0"
I20260807 05:11:41.558979 21055 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 05:11:41.578826 21072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:41.580124 21055 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:11:41.580258 21055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "e2835474566449cca400fb5945831651"
format_stamp: "Formatted at 2026-08-07 05:11:41 on dist-test-slave-49h3"
server_key: "e484965bbe4475ab86dcd178365d4982"
server_key_iv: "6f9be70792062950732591ef936d3fc9"
server_key_version: "kuduclusterkey@0"
I20260807 05:11:41.580380 21055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:41.604239 21055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:41.607782 21055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:41.608072 21055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:41.608844 21055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:11:41.609965 21055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:11:41.610071 21055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:41.610188 21055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:11:41.610257 21055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:41.622046 21055 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:39619
I20260807 05:11:41.623356 21055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:11:41.624718 21194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:39619 every 8 connection(s)
I20260807 05:11:41.630303 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 21055
I20260807 05:11:41.630434 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:11:41.630774 8257 external_mini_cluster.cc:1428] Setting key ceaebc71946e5f81acf6fb521c7763a8
Aug 07 05:11:41 dist-test-slave-49h3 krb5kdc[20467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079501, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.16.66@KRBTEST.COM; defaulting to no policy
I20260807 05:11:41.646118 21198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:41.628799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:48213 (local address 127.8.16.126:34391)
0807 05:11:41.629726 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:41.629736 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:41.630440 (+ 704us) server_negotiation.cc:407] Connection header received
0807 05:11:41.634997 (+ 4557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:41.635000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:41.635059 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:41.635139 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:41.636765 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:41.637623 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:41.639074 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:41.639300 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:41.639384 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:41.642133 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:41.642136 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:41.642138 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:41.642167 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:41.643651 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:41.644217 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:41.644220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:41.644222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:41.644314 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:41.644734 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:41.644737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:41.644740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:41.644985 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:41.645122 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:41.645768 (+ 646us) server_negotiation.cc:299] Negotiation successful
0807 05:11:41.645910 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":840,"thread_start_us":92,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.66@KRBTEST.COM" created.
I20260807 05:11:41.646934 21197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:41.627652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:34391 (local address 127.8.16.65:48213)
0807 05:11:41.629988 (+ 2336us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:41.630045 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:41.631287 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:41.635508 (+ 4221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:41.635518 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:41.635903 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:41.636604 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:41.636622 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:41.637779 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:41.637783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:41.638944 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:41.638954 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:41.639455 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:41.639460 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:41.639660 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:41.640314 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:41.640336 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:41.641953 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:41.643771 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:41.643777 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:41.643794 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:41.644122 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:41.644396 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:41.644399 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:41.644401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:41.644608 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:41.645135 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:41.645151 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:41.645515 (+ 364us) client_negotiation.cc:769] Sending connection context
0807 05:11:41.645692 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 05:11:41.645979 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2122,"thread_start_us":159,"threads_started":1}
I20260807 05:11:41.648324 21195 heartbeater.cc:347] Connected to a master server at 127.8.16.126:34391
I20260807 05:11:41.648599 21195 heartbeater.cc:464] Registering TS with master...
I20260807 05:11:41.649181 21195 heartbeater.cc:511] Master 127.8.16.126:34391 requested a full tablet report, sending...
I20260807 05:11:41.650924 20969 ts_manager.cc:194] Registered new tserver with Master: e2835474566449cca400fb5945831651 (127.8.16.65:39619)
I20260807 05:11:41.652496 20969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:48213
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
I20260807 05:11:41.662209 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.16.84:59173/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:34391
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 05:11:41 dist-test-slave-49h3 krb5kdc[20467](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786079501, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 05:11:41.799899 21204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:41.800163 21204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:41.800221 21204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:41.804558 21204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:41.804625 21204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:41.804702 21204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:11:41.810076 21204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.16.84:59173/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:34391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:41.811105 21204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:41.812173 21204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:11:41.821480 21211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:41.821480 21213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:41.822094 21210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:41.822578 21204 server_base.cc:1034] running on GCE node
I20260807 05:11:41.822974 21204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:41.823673 21204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:41.824842 21204 hybrid_clock.cc:648] HybridClock initialized: now 1786079501824818 us; error 38 us; skew 500 ppm
Aug 07 05:11:41 dist-test-slave-49h3 krb5kdc[20467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079501, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:41.827479 21204 init.cc:377] Logged in from keytab as kudu/127.8.16.66@KRBTEST.COM (short username kudu)
I20260807 05:11:41.828329 21204 webserver.cc:460] Webserver started at http://127.8.16.66:42393/ using document root <none> and password file <none>
I20260807 05:11:41.828920 21204 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:41.828989 21204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:11:41.829201 21204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 05:11:41 dist-test-slave-49h3 krb5kdc[20467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079501, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for HTTP/127.8.16.84@KRBTEST.COM
I20260807 05:11:41.843616 21204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "475d75f5c58d426ebd308e381a49f4bc"
format_stamp: "Formatted at 2026-08-07 05:11:41 on dist-test-slave-49h3"
server_key: "100676c8daccec841408521fdf9d700f"
server_key_iv: "d6d37626a8b7e135a02f753159ef1832"
server_key_version: "kuduclusterkey@0"
I20260807 05:11:41.844405 21204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "475d75f5c58d426ebd308e381a49f4bc"
format_stamp: "Formatted at 2026-08-07 05:11:41 on dist-test-slave-49h3"
server_key: "100676c8daccec841408521fdf9d700f"
server_key_iv: "d6d37626a8b7e135a02f753159ef1832"
server_key_version: "kuduclusterkey@0"
I20260807 05:11:41.853901 21204 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.000s
I20260807 05:11:41.873502 21220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:41.879325 21204 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.003s sys 0.000s
I20260807 05:11:41.879580 21204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "475d75f5c58d426ebd308e381a49f4bc"
format_stamp: "Formatted at 2026-08-07 05:11:41 on dist-test-slave-49h3"
server_key: "100676c8daccec841408521fdf9d700f"
server_key_iv: "d6d37626a8b7e135a02f753159ef1832"
server_key_version: "kuduclusterkey@0"
I20260807 05:11:41.879851 21204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:41.901892 21204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:41.905994 21204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:41.906277 21204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:41.907087 21204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:11:41.909107 21204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:11:41.909173 21204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:41.909226 21204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:11:41.909247 21204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:41.922825 21204 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:34279
I20260807 05:11:41.924046 21204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:11:41.925489 21342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:34279 every 8 connection(s)
I20260807 05:11:41.933753 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 21204
I20260807 05:11:41.933893 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:11:41.934247 8257 external_mini_cluster.cc:1428] Setting key 3a2c5ce2f0e6c6ae3e227835f5b75a25
Aug 07 05:11:41 dist-test-slave-49h3 krb5kdc[20467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079501, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:11:41.953246 21198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:41.927685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:54349 (local address 127.8.16.126:34391)
0807 05:11:41.929254 (+ 1569us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:41.929259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:41.929757 (+ 498us) server_negotiation.cc:407] Connection header received
0807 05:11:41.931006 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:41.931010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:41.931067 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:41.931122 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:41.932759 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:41.933625 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:41.935268 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:41.935723 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:41.935816 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:41.938818 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:41.938822 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:41.938825 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:41.938857 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:41.947199 (+ 8342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:41.950999 (+ 3800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:41.951005 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:41.951007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:41.951139 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:41.951641 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:41.951645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:41.951647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:41.951905 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:41.951994 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:41.952906 (+ 912us) server_negotiation.cc:299] Negotiation successful
0807 05:11:41.953056 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1464}
I20260807 05:11:41.955622 21347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:41.928200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:34391 (local address 127.8.16.66:54349)
0807 05:11:41.929587 (+ 1387us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:41.929647 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:41.930750 (+ 1103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:41.931527 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:41.931536 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:41.931911 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:41.932584 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:41.932602 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:41.933796 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:41.933799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:41.935145 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:41.935155 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:41.936021 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:41.936026 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:41.936224 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:41.936881 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:41.936901 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:41.938593 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:41.948256 (+ 9663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:41.948265 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:41.948286 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:41.948692 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:41.951271 (+ 2579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:41.951275 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:41.951277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:41.951516 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:41.952086 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:41.952093 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:41.952493 (+ 400us) client_negotiation.cc:769] Sending connection context
0807 05:11:41.954036 (+ 1543us) client_negotiation.cc:240] Negotiation successful
0807 05:11:41.954441 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1138,"thread_start_us":153,"threads_started":1}
WARNING: no policy specified for kudu/127.8.16.67@KRBTEST.COM; defaulting to no policy
I20260807 05:11:41.957567 21344 heartbeater.cc:347] Connected to a master server at 127.8.16.126:34391
I20260807 05:11:41.957841 21344 heartbeater.cc:464] Registering TS with master...
I20260807 05:11:41.958436 21344 heartbeater.cc:511] Master 127.8.16.126:34391 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.67@KRBTEST.COM" created.
I20260807 05:11:41.968238 20969 ts_manager.cc:194] Registered new tserver with Master: 475d75f5c58d426ebd308e381a49f4bc (127.8.16.66:34279)
I20260807 05:11:41.969048 20969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:54349
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
I20260807 05:11:41.986493 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.16.84:59173/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:34391
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:11:42.149889 21350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:42.150411 21350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:42.150560 21350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:42.157609 21350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:42.157745 21350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:42.157878 21350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:11:42.165740 21350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.16.84:59173/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:34391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:42.167371 21350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:42.168843 21350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:11:42.182253 21355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:42.182569 21350 server_base.cc:1034] running on GCE node
W20260807 05:11:42.182698 21356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:42.183279 21359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:42.183849 21350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:42.184631 21350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:42.185784 21350 hybrid_clock.cc:648] HybridClock initialized: now 1786079502185713 us; error 86 us; skew 500 ppm
Aug 07 05:11:42 dist-test-slave-49h3 krb5kdc[20467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079502, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:42.189471 21350 init.cc:377] Logged in from keytab as kudu/127.8.16.67@KRBTEST.COM (short username kudu)
I20260807 05:11:42.190565 21350 webserver.cc:460] Webserver started at http://127.8.16.67:33197/ using document root <none> and password file <none>
I20260807 05:11:42.191275 21350 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:42.191352 21350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:11:42.191572 21350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 05:11:42 dist-test-slave-49h3 krb5kdc[20467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079502, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for HTTP/127.8.16.84@KRBTEST.COM
I20260807 05:11:42.210928 21350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "6e19e54494b449b2ba81ea45bf5bf6b3"
format_stamp: "Formatted at 2026-08-07 05:11:42 on dist-test-slave-49h3"
server_key: "5c7894af64219432928c3239331598ae"
server_key_iv: "ab4f0624845bd79917157f38402843b8"
server_key_version: "kuduclusterkey@0"
I20260807 05:11:42.212795 21350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "6e19e54494b449b2ba81ea45bf5bf6b3"
format_stamp: "Formatted at 2026-08-07 05:11:42 on dist-test-slave-49h3"
server_key: "5c7894af64219432928c3239331598ae"
server_key_iv: "ab4f0624845bd79917157f38402843b8"
server_key_version: "kuduclusterkey@0"
I20260807 05:11:42.218657 21350 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 05:11:42.244874 21366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:42.246654 21350 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 05:11:42.246790 21350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "6e19e54494b449b2ba81ea45bf5bf6b3"
format_stamp: "Formatted at 2026-08-07 05:11:42 on dist-test-slave-49h3"
server_key: "5c7894af64219432928c3239331598ae"
server_key_iv: "ab4f0624845bd79917157f38402843b8"
server_key_version: "kuduclusterkey@0"
I20260807 05:11:42.246925 21350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:42.265992 21350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:42.269814 21350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:42.270108 21350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:42.270865 21350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:11:42.271994 21350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:11:42.272117 21350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:42.272215 21350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:11:42.272285 21350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:42.283370 21350 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:42125
I20260807 05:11:42.288530 21350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:11:42.287295 21488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:42125 every 8 connection(s)
I20260807 05:11:42.293952 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 21350
I20260807 05:11:42.294085 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:11:42.294390 8257 external_mini_cluster.cc:1428] Setting key 7652be854e0bbe18b8a61813193fb284
Aug 07 05:11:42 dist-test-slave-49h3 krb5kdc[20467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079502, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:11:42.309490 21198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:42.292984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:38143 (local address 127.8.16.126:34391)
0807 05:11:42.293147 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:42.293152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:42.293498 (+ 346us) server_negotiation.cc:407] Connection header received
0807 05:11:42.294771 (+ 1273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:42.294774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:42.294831 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:42.294887 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:42.297869 (+ 2982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:42.298724 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:42.300422 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:42.300748 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:42.300830 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:42.303800 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:42.303804 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:42.303808 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:42.303838 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:42.305726 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:42.306367 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:42.306371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:42.306373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:42.306458 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:42.306834 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:42.306837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:42.306839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:42.307060 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:42.307148 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:42.309151 (+ 2003us) server_negotiation.cc:299] Negotiation successful
0807 05:11:42.309288 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 05:11:42.311440 21491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:42.291867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:34391 (local address 127.8.16.67:38143)
0807 05:11:42.293316 (+ 1449us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:42.293372 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:42.294513 (+ 1141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:42.296569 (+ 2056us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:42.296580 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:42.296956 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:42.297693 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:42.297715 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:42.298878 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:42.298881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:42.300286 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:42.300297 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:42.301046 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:42.301051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:42.301307 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:42.301855 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:42.301875 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:42.303587 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:42.305852 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:42.305858 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:42.305874 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:42.306252 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:42.306555 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:42.306558 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:42.306560 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:42.306732 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:42.307217 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:42.307223 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:42.308853 (+ 1630us) client_negotiation.cc:769] Sending connection context
0807 05:11:42.309943 (+ 1090us) client_negotiation.cc:240] Negotiation successful
0807 05:11:42.310314 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1237,"thread_start_us":161,"threads_started":1}
I20260807 05:11:42.312942 21489 heartbeater.cc:347] Connected to a master server at 127.8.16.126:34391
I20260807 05:11:42.313197 21489 heartbeater.cc:464] Registering TS with master...
I20260807 05:11:42.313807 21489 heartbeater.cc:511] Master 127.8.16.126:34391 requested a full tablet report, sending...
I20260807 05:11:42.315004 20969 ts_manager.cc:194] Registered new tserver with Master: 6e19e54494b449b2ba81ea45bf5bf6b3 (127.8.16.67:42125)
I20260807 05:11:42.315737 20969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:38143
I20260807 05:11:42.320418 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:11:42.321174 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 21055
Aug 07 05:11:42 dist-test-slave-49h3 krb5kdc[20467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079501, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for HTTP/127.8.16.84@KRBTEST.COM
I20260807 05:11:42.329347 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 21204
I20260807 05:11:42.340205 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 21350
I20260807 05:11:42.353375 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 20936
I20260807 05:11:42.366586 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:34391
--webserver_interface=127.8.16.126
--webserver_port=39809
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:34391
--ranger_config_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.16.84:59173/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.16.84:59173/kms with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:11:42.484090 21492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:42.484344 21492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:42.484385 21492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:42.488346 21492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:11:42.488409 21492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:42.488426 21492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:42.488441 21492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:11:42.488454 21492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:11:42.493773 21492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.16.84:59173/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:34391
--ranger_config_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:34391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=39809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:42.494727 21492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:42.495671 21492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:11:42.504112 21498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:42.504119 21497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:42.504299 21492 server_base.cc:1034] running on GCE node
W20260807 05:11:42.504119 21500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:42.504987 21492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:42.506196 21492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:42.507375 21492 hybrid_clock.cc:648] HybridClock initialized: now 1786079502507358 us; error 35 us; skew 500 ppm
Aug 07 05:11:42 dist-test-slave-49h3 krb5kdc[20467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079502, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:42.510109 21492 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:11:42.510967 21492 webserver.cc:460] Webserver started at http://127.8.16.126:39809/ using document root <none> and password file <none>
I20260807 05:11:42.511478 21492 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:42.511549 21492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 05:11:42 dist-test-slave-49h3 krb5kdc[20467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079502, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for HTTP/127.8.16.84@KRBTEST.COM
I20260807 05:11:42.552465 21492 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:11:42.554639 21508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:42.555850 21492 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:11:42.555968 21492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "bc93185fc657494f926432313e6c2530"
format_stamp: "Formatted at 2026-08-07 05:11:40 on dist-test-slave-49h3"
server_key: "78117fa71823014ff3e9f0096be10398"
server_key_iv: "f38ee6516d1c8c21482de0ad6788e224"
server_key_version: "kuduclusterkey@0"
I20260807 05:11:42.556499 21492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:42.576030 21492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:42.578765 21492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:42.578938 21492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:42.585758 21492 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:34391
I20260807 05:11:42.585780 21562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:34391 every 8 connection(s)
I20260807 05:11:42.586830 21492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:11:42.590849 21563 sys_catalog.cc:263] Verifying existing consensus state
I20260807 05:11:42.592401 21563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530: Bootstrap starting.
I20260807 05:11:42.595193 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 21492
I20260807 05:11:42.595598 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.16.84:59173/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:39619
--local_ip_for_outbound_sockets=127.8.16.65
--tserver_master_addrs=127.8.16.126:34391
--webserver_port=35691
--webserver_interface=127.8.16.65
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.16.84:59173/kms with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:11:42.608201 21563 log.cc:824] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530: Log is configured to *not* fsync() on all Append() calls
I20260807 05:11:42.614710 21563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:11:42.615058 21563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530: Bootstrap complete.
I20260807 05:11:42.618925 21563 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc93185fc657494f926432313e6c2530" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 34391 } }
I20260807 05:11:42.619303 21563 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc93185fc657494f926432313e6c2530, State: Initialized, Role: FOLLOWER
I20260807 05:11:42.619772 21563 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc93185fc657494f926432313e6c2530" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 34391 } }
I20260807 05:11:42.619894 21563 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:11:42.619958 21563 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:11:42.620050 21563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:11:42.621886 21563 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc93185fc657494f926432313e6c2530" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 34391 } }
I20260807 05:11:42.622176 21563 leader_election.cc:302] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc93185fc657494f926432313e6c2530; no voters:
I20260807 05:11:42.622468 21563 leader_election.cc:288] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 05:11:42.622599 21567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 05:11:42.622799 21567 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [term 2 LEADER]: Becoming Leader. State: Replica: bc93185fc657494f926432313e6c2530, State: Running, Role: LEADER
I20260807 05:11:42.623153 21567 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc93185fc657494f926432313e6c2530" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 34391 } }
I20260807 05:11:42.623647 21563 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:11:42.624025 21567 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc93185fc657494f926432313e6c2530. Latest consensus state: current_term: 2 leader_uuid: "bc93185fc657494f926432313e6c2530" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc93185fc657494f926432313e6c2530" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 34391 } } }
I20260807 05:11:42.624215 21567 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [sys.catalog]: This master's current role is: LEADER
I20260807 05:11:42.624425 21568 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bc93185fc657494f926432313e6c2530" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc93185fc657494f926432313e6c2530" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 34391 } } }
I20260807 05:11:42.624542 21568 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530 [sys.catalog]: This master's current role is: LEADER
I20260807 05:11:42.624717 21572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:11:42.627825 21572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:11:42.628328 21563 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 05:11:42.628568 21572 catalog_manager.cc:1259] Loaded cluster ID: eb0da21a31d24952a1b4d1bd6b606bf5
I20260807 05:11:42.628618 21572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:11:42.630964 21572 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:11:42.631618 21572 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530: Loaded TSK: 0
I20260807 05:11:42.632774 21572 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 05:11:42.732973 21565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:42.733304 21565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:42.733369 21565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:42.739578 21565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:42.739733 21565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:42.739843 21565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:11:42.747759 21565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.16.84:59173/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:39619
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=35691
--tserver_master_addrs=127.8.16.126:34391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:42.749214 21565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:42.750710 21565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:11:42.763330 21604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:42.763343 21602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:42.763465 21601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:42.764386 21565 server_base.cc:1034] running on GCE node
I20260807 05:11:42.764889 21565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:42.765767 21565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:42.766947 21565 hybrid_clock.cc:648] HybridClock initialized: now 1786079502766925 us; error 33 us; skew 500 ppm
Aug 07 05:11:42 dist-test-slave-49h3 krb5kdc[20467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079502, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:42.770666 21565 init.cc:377] Logged in from keytab as kudu/127.8.16.65@KRBTEST.COM (short username kudu)
I20260807 05:11:42.771795 21565 webserver.cc:460] Webserver started at http://127.8.16.65:35691/ using document root <none> and password file <none>
I20260807 05:11:42.772548 21565 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:42.772729 21565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 05:11:42 dist-test-slave-49h3 krb5kdc[20467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079502, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for HTTP/127.8.16.84@KRBTEST.COM
I20260807 05:11:42.795435 21565 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.000s
I20260807 05:11:42.798240 21613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:42.799554 21565 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 05:11:42.799675 21565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "e2835474566449cca400fb5945831651"
format_stamp: "Formatted at 2026-08-07 05:11:41 on dist-test-slave-49h3"
server_key: "e484965bbe4475ab86dcd178365d4982"
server_key_iv: "6f9be70792062950732591ef936d3fc9"
server_key_version: "kuduclusterkey@0"
I20260807 05:11:42.800273 21565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:42.818687 21565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:42.821652 21565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:42.821841 21565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:42.822533 21565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:11:42.823544 21565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:11:42.823593 21565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:42.823644 21565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:11:42.823665 21565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:42.834260 21565 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:39619
I20260807 05:11:42.834496 21731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:39619 every 8 connection(s)
I20260807 05:11:42.835644 21565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:11:42.836815 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 21565
I20260807 05:11:42.837332 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.16.84:59173/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:34279
--local_ip_for_outbound_sockets=127.8.16.66
--tserver_master_addrs=127.8.16.126:34391
--webserver_port=42393
--webserver_interface=127.8.16.66
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.16.84:59173/kms with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 05:11:42 dist-test-slave-49h3 krb5kdc[20467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079502, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:11:42.856534 21735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:42.838294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:34391 (local address 127.8.16.65:38683)
0807 05:11:42.838891 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:42.838955 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:42.840021 (+ 1066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:42.842038 (+ 2017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:42.842047 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:42.842401 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:42.843111 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:42.843132 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:42.845419 (+ 2287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:42.845424 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:42.846890 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:42.846903 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:42.848000 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:42.848005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:42.848202 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:42.848943 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:42.848966 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:42.850759 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:42.853127 (+ 2368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:42.853133 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:42.853147 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:42.853510 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:42.853904 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:42.853908 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:42.853910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:42.854068 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:42.854760 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:42.854766 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:42.855059 (+ 293us) client_negotiation.cc:769] Sending connection context
0807 05:11:42.855231 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 05:11:42.855543 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":137,"threads_started":1}
I20260807 05:11:42.860283 21737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:42.840228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:38683 (local address 127.8.16.126:34391)
0807 05:11:42.840753 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:42.840774 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:42.840822 (+ 48us) server_negotiation.cc:407] Connection header received
0807 05:11:42.841113 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:42.841127 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:42.841567 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:42.843382 (+ 1815us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:42.843730 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:42.845248 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:42.847052 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:42.847804 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:42.849784 (+ 1980us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:42.850974 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:42.850981 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:42.850996 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:42.851027 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:42.852999 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:42.853648 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:42.853655 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:42.853660 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:42.853802 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:42.854170 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:42.854173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:42.854175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:42.854620 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:42.857547 (+ 2927us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:42.857723 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 05:11:42.858030 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":133,"threads_started":1}
I20260807 05:11:42.864837 21733 heartbeater.cc:347] Connected to a master server at 127.8.16.126:34391
I20260807 05:11:42.865113 21733 heartbeater.cc:464] Registering TS with master...
I20260807 05:11:42.865739 21733 heartbeater.cc:511] Master 127.8.16.126:34391 requested a full tablet report, sending...
W20260807 05:11:42.868180 21733 heartbeater.cc:644] Failed to heartbeat to 127.8.16.126:34391 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 05:11:42.868300 21733 heartbeater.cc:464] Registering TS with master...
I20260807 05:11:42.868475 21733 heartbeater.cc:511] Master 127.8.16.126:34391 requested a full tablet report, sending...
I20260807 05:11:42.869498 21733 heartbeater.cc:464] Registering TS with master...
I20260807 05:11:42.869665 21733 heartbeater.cc:511] Master 127.8.16.126:34391 requested a full tablet report, sending...
W20260807 05:11:42.870527 21733 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 05:11:42.980590 21736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:42.980953 21736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:42.981017 21736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:42.986905 21736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:42.986974 21736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:42.987033 21736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:11:42.992856 21736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.16.84:59173/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:34279
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=42393
--tserver_master_addrs=127.8.16.126:34391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:42.993973 21736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:42.995004 21736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:11:43.004236 21742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:43.004281 21743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:43.004706 21736 server_base.cc:1034] running on GCE node
W20260807 05:11:43.004393 21745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:43.008813 21736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:43.009644 21736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:43.010848 21736 hybrid_clock.cc:648] HybridClock initialized: now 1786079503010817 us; error 39 us; skew 500 ppm
Aug 07 05:11:43 dist-test-slave-49h3 krb5kdc[20467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079503, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:43.014338 21736 init.cc:377] Logged in from keytab as kudu/127.8.16.66@KRBTEST.COM (short username kudu)
I20260807 05:11:43.015450 21736 webserver.cc:460] Webserver started at http://127.8.16.66:42393/ using document root <none> and password file <none>
I20260807 05:11:43.016130 21736 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:43.016237 21736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 05:11:43 dist-test-slave-49h3 krb5kdc[20467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079503, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for HTTP/127.8.16.84@KRBTEST.COM
I20260807 05:11:43.038909 21736 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.003s sys 0.000s
I20260807 05:11:43.042181 21753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:43.043864 21736 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 05:11:43.044016 21736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "475d75f5c58d426ebd308e381a49f4bc"
format_stamp: "Formatted at 2026-08-07 05:11:41 on dist-test-slave-49h3"
server_key: "100676c8daccec841408521fdf9d700f"
server_key_iv: "d6d37626a8b7e135a02f753159ef1832"
server_key_version: "kuduclusterkey@0"
I20260807 05:11:43.044793 21736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:43.069550 21736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:43.073215 21736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:43.073495 21736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:43.074262 21736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:11:43.075371 21736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:11:43.075476 21736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:43.075567 21736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:11:43.075621 21736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:43.088102 21736 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:34279
I20260807 05:11:43.089592 21736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:11:43.089874 21878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:34279 every 8 connection(s)
I20260807 05:11:43.101718 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 21736
Aug 07 05:11:43 dist-test-slave-49h3 krb5kdc[20467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079503, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:11:43.102160 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.16.84:59173/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:42125
--local_ip_for_outbound_sockets=127.8.16.67
--tserver_master_addrs=127.8.16.126:34391
--webserver_port=33197
--webserver_interface=127.8.16.67
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.16.84:59173/kms with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:11:43.116602 21737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:43.092943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:37229 (local address 127.8.16.126:34391)
0807 05:11:43.093105 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:43.093110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:43.093986 (+ 876us) server_negotiation.cc:407] Connection header received
0807 05:11:43.096273 (+ 2287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:43.096277 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:43.096332 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:43.096405 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:43.097798 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:43.098519 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:43.099641 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:43.099872 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:43.099960 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:43.104306 (+ 4346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:43.104310 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:43.104313 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:43.104345 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:43.105928 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:43.114768 (+ 8840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:43.114772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:43.114774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:43.114875 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:43.115266 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:43.115270 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:43.115272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:43.115498 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:43.115578 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:43.116267 (+ 689us) server_negotiation.cc:299] Negotiation successful
0807 05:11:43.116425 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 05:11:43.118728 21881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:43.093345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:34391 (local address 127.8.16.66:37229)
0807 05:11:43.093835 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:43.093887 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:43.096053 (+ 2166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:43.096569 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:43.096576 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:43.096856 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:43.097623 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:43.097644 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:43.098660 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:43.098663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:43.099508 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:43.099517 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:43.100012 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:43.100017 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:43.100178 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:43.100734 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:43.100751 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:43.104076 (+ 3325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:43.114195 (+ 10119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:43.114203 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:43.114222 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:43.114613 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:43.114979 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:43.114982 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:43.114984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:43.115160 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:43.115654 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:43.115660 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:43.115994 (+ 334us) client_negotiation.cc:769] Sending connection context
0807 05:11:43.117226 (+ 1232us) client_negotiation.cc:240] Negotiation successful
0807 05:11:43.117701 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":122,"threads_started":1}
I20260807 05:11:43.120234 21879 heartbeater.cc:347] Connected to a master server at 127.8.16.126:34391
I20260807 05:11:43.120468 21879 heartbeater.cc:464] Registering TS with master...
I20260807 05:11:43.121012 21879 heartbeater.cc:511] Master 127.8.16.126:34391 requested a full tablet report, sending...
W20260807 05:11:43.122720 21879 heartbeater.cc:644] Failed to heartbeat to 127.8.16.126:34391 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 05:11:43.122833 21879 heartbeater.cc:464] Registering TS with master...
I20260807 05:11:43.122984 21879 heartbeater.cc:511] Master 127.8.16.126:34391 requested a full tablet report, sending...
I20260807 05:11:43.123936 21879 heartbeater.cc:464] Registering TS with master...
I20260807 05:11:43.124099 21879 heartbeater.cc:511] Master 127.8.16.126:34391 requested a full tablet report, sending...
W20260807 05:11:43.125126 21879 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 05:11:43.246845 21882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:43.247315 21882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:43.247439 21882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:43.253506 21882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:43.253655 21882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:43.253784 21882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:11:43.261744 21882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.16.84:59173/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:42125
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=33197
--tserver_master_addrs=127.8.16.126:34391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:43.263334 21882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:43.264787 21882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:11:43.277066 21890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:43.277072 21889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:43.277056 21892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:43.278311 21882 server_base.cc:1034] running on GCE node
I20260807 05:11:43.278812 21882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:43.279601 21882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:43.280794 21882 hybrid_clock.cc:648] HybridClock initialized: now 1786079503280755 us; error 51 us; skew 500 ppm
Aug 07 05:11:43 dist-test-slave-49h3 krb5kdc[20467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079503, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:43.284163 21882 init.cc:377] Logged in from keytab as kudu/127.8.16.67@KRBTEST.COM (short username kudu)
I20260807 05:11:43.285115 21882 webserver.cc:460] Webserver started at http://127.8.16.67:33197/ using document root <none> and password file <none>
I20260807 05:11:43.285709 21882 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:43.285795 21882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 05:11:43 dist-test-slave-49h3 krb5kdc[20467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079503, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for HTTP/127.8.16.84@KRBTEST.COM
I20260807 05:11:43.311676 21882 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.000s
I20260807 05:11:43.314534 21900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:43.315919 21882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:11:43.316044 21882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "6e19e54494b449b2ba81ea45bf5bf6b3"
format_stamp: "Formatted at 2026-08-07 05:11:42 on dist-test-slave-49h3"
server_key: "5c7894af64219432928c3239331598ae"
server_key_iv: "ab4f0624845bd79917157f38402843b8"
server_key_version: "kuduclusterkey@0"
I20260807 05:11:43.316840 21882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:43.330353 21882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:43.333550 21882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:43.333737 21882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:43.334354 21882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:11:43.335219 21882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:11:43.335278 21882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:43.335332 21882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:11:43.335357 21882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:43.345925 21882 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:42125
I20260807 05:11:43.347787 21882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:11:43.351239 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 21882
I20260807 05:11:43.354457 22026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:42125 every 8 connection(s)
Aug 07 05:11:43 dist-test-slave-49h3 krb5kdc[20467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079503, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:11:43.365974 21737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:43.352425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58436 (local address 127.8.16.126:34391)
0807 05:11:43.352563 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:43.352568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:43.355928 (+ 3360us) server_negotiation.cc:407] Connection header received
0807 05:11:43.356053 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:43.356057 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:43.356114 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:43.356224 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:43.357530 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:43.358364 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:43.360012 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:43.360377 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:43.360510 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:43.362156 (+ 1646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:43.362160 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:43.362163 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:43.362191 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:43.363910 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:43.364375 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:43.364379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:43.364381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:43.364474 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:43.364785 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:43.364789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:43.364791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:43.364966 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:43.365174 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:43.365700 (+ 526us) server_negotiation.cc:299] Negotiation successful
0807 05:11:43.365815 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 05:11:43.369150 22031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:43.354187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:36391 (local address 127.8.16.126:34391)
0807 05:11:43.357063 (+ 2876us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:43.357078 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:43.357101 (+ 23us) server_negotiation.cc:407] Connection header received
0807 05:11:43.357139 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:43.357141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:43.357185 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:43.359747 (+ 2562us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:43.359842 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:43.360511 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:43.362169 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:43.362379 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:43.362467 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:43.365099 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:43.365103 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:43.365111 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:43.365133 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:43.367055 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:43.367605 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:43.367609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:43.367610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:43.367692 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:43.368027 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:43.368030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:43.368032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:43.368220 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:43.368306 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:43.368819 (+ 513us) server_negotiation.cc:299] Negotiation successful
0807 05:11:43.368936 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1919,"thread_start_us":92,"threads_started":1}
I20260807 05:11:43.370012 22030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:43.353055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:34391 (local address 127.8.16.67:36391)
0807 05:11:43.354338 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:43.354391 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:43.355381 (+ 990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:43.357504 (+ 2123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:43.357513 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:43.357832 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:43.358470 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:43.358490 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:43.360651 (+ 2161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:43.360654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:43.362023 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:43.362032 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:43.362561 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:43.362565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:43.362775 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:43.363447 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:43.363463 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:43.364916 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:43.367161 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:43.367167 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:43.367190 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:43.367505 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:43.367770 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:43.367774 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:43.367776 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:43.367938 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:43.368319 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:43.368326 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:43.368586 (+ 260us) client_negotiation.cc:769] Sending connection context
0807 05:11:43.368746 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 05:11:43.369065 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1070,"thread_start_us":124,"threads_started":1}
I20260807 05:11:43.371083 22027 heartbeater.cc:347] Connected to a master server at 127.8.16.126:34391
I20260807 05:11:43.371330 22027 heartbeater.cc:464] Registering TS with master...
I20260807 05:11:43.371804 22027 heartbeater.cc:511] Master 127.8.16.126:34391 requested a full tablet report, sending...
W20260807 05:11:43.373260 22027 heartbeater.cc:644] Failed to heartbeat to 127.8.16.126:34391 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 05:11:43.373374 22027 heartbeater.cc:464] Registering TS with master...
I20260807 05:11:43.373559 22027 heartbeater.cc:511] Master 127.8.16.126:34391 requested a full tablet report, sending...
I20260807 05:11:43.374329 22027 heartbeater.cc:464] Registering TS with master...
I20260807 05:11:43.374488 22027 heartbeater.cc:511] Master 127.8.16.126:34391 requested a full tablet report, sending...
W20260807 05:11:43.375375 22027 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 05:11:43 dist-test-slave-49h3 krb5kdc[20467](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786079503, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:43.870743 21733 heartbeater.cc:464] Registering TS with master...
I20260807 05:11:43.871050 21733 heartbeater.cc:511] Master 127.8.16.126:34391 requested a full tablet report, sending...
I20260807 05:11:43.872282 21525 ts_manager.cc:194] Registered new tserver with Master: e2835474566449cca400fb5945831651 (127.8.16.65:39619)
I20260807 05:11:43.873771 21525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:38683
Aug 07 05:11:44 dist-test-slave-49h3 krb5kdc[20467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079503, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for HTTP/127.8.16.84@KRBTEST.COM
I20260807 05:11:44.125342 21879 heartbeater.cc:464] Registering TS with master...
I20260807 05:11:44.125768 21879 heartbeater.cc:511] Master 127.8.16.126:34391 requested a full tablet report, sending...
I20260807 05:11:44.126523 21525 ts_manager.cc:194] Registered new tserver with Master: 475d75f5c58d426ebd308e381a49f4bc (127.8.16.66:34279)
I20260807 05:11:44.127319 21525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:37229
Aug 07 05:11:44 dist-test-slave-49h3 krb5kdc[20467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079503, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for HTTP/127.8.16.84@KRBTEST.COM
I20260807 05:11:44.375610 22027 heartbeater.cc:464] Registering TS with master...
I20260807 05:11:44.375934 22027 heartbeater.cc:511] Master 127.8.16.126:34391 requested a full tablet report, sending...
I20260807 05:11:44.376751 21525 ts_manager.cc:194] Registered new tserver with Master: 6e19e54494b449b2ba81ea45bf5bf6b3 (127.8.16.67:42125)
I20260807 05:11:44.377735 21525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:36391
I20260807 05:11:44.383404 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:11:44.395915 22050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:44.385288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58452 (local address 127.8.16.126:34391)
0807 05:11:44.385887 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:44.385891 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:44.386415 (+ 524us) server_negotiation.cc:407] Connection header received
0807 05:11:44.386460 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:44.386463 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:44.386514 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:44.386587 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:44.387511 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.388361 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.389611 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.389975 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.390075 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:44.391861 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:44.391865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:44.391868 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:44.391896 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:44.393663 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:44.394161 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:44.394165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:44.394167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:44.394259 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:44.394559 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:44.394562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:44.394564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:44.394779 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:44.394925 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:44.395578 (+ 653us) server_negotiation.cc:299] Negotiation successful
0807 05:11:44.395710 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":112,"threads_started":1}
I20260807 05:11:44.400864 21525 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58452:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:11:44.406132 21525 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:11:44.468676 22056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:44.453778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:34279 (local address 127.0.0.1:40192)
0807 05:11:44.461611 (+ 7833us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:44.461626 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:44.461733 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:44.463036 (+ 1303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:44.463039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:44.463056 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:11:44.463146 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:44.463151 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.464561 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.464565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:44.465859 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:44.465871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.468070 (+ 2199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.468074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:44.468167 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:44.468243 (+ 76us) client_negotiation.cc:769] Sending connection context
0807 05:11:44.468364 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 05:11:44.468498 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7762,"mutex_wait_us":19,"thread_start_us":88,"threads_started":1}
I20260807 05:11:44.469360 22058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:44.453388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:42125 (local address 127.0.0.1:45124)
0807 05:11:44.456276 (+ 2888us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:44.456289 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:44.456388 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:44.462457 (+ 6069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:44.462460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:44.462478 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:11:44.462567 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:44.462573 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.465809 (+ 3236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.465813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:44.466978 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:44.466989 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.468102 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.468106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:44.469018 (+ 912us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:44.469045 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 05:11:44.469125 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 05:11:44.469199 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2825,"mutex_wait_us":794,"thread_start_us":124,"threads_started":1}
I20260807 05:11:44.470381 22061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:44.454067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40192 (local address 127.8.16.66:34279)
0807 05:11:44.455555 (+ 1488us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:44.455563 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:44.461731 (+ 6168us) server_negotiation.cc:407] Connection header received
0807 05:11:44.462716 (+ 985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:44.462722 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:44.462910 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:44.463058 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:11:44.463276 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.464427 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.466007 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.467742 (+ 1735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.467849 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:44.468002 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:44.470114 (+ 2112us) server_negotiation.cc:299] Negotiation successful
0807 05:11:44.470227 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1366,"thread_start_us":137,"threads_started":1}
I20260807 05:11:44.473886 21805 tablet_service.cc:1467] Processing CreateTablet for tablet bcb13c736b7343b78a8d2c125d056813 (DEFAULT_TABLE table=test-table [id=f2e0b86714944e6ba5ae04aad9e436e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:11:44.475152 21805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcb13c736b7343b78a8d2c125d056813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:11:44.477821 22059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:44.453571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45124 (local address 127.8.16.67:42125)
0807 05:11:44.461967 (+ 8396us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:44.461974 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:44.461996 (+ 22us) server_negotiation.cc:407] Connection header received
0807 05:11:44.462067 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:44.462083 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:44.462302 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:44.462720 (+ 418us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:11:44.462866 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.465678 (+ 2812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.467147 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.467951 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.468231 (+ 280us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:44.468368 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:44.477545 (+ 9177us) server_negotiation.cc:299] Negotiation successful
0807 05:11:44.477676 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8282,"thread_start_us":79,"threads_started":1}
I20260807 05:11:44.480850 21933 tablet_service.cc:1467] Processing CreateTablet for tablet bcb13c736b7343b78a8d2c125d056813 (DEFAULT_TABLE table=test-table [id=f2e0b86714944e6ba5ae04aad9e436e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:11:44.481920 21933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcb13c736b7343b78a8d2c125d056813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:11:44.483518 22060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:44.450559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:39619 (local address 127.0.0.1:35740)
0807 05:11:44.455833 (+ 5274us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:44.455898 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:44.456039 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:44.458445 (+ 2406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:44.458452 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:44.458479 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:11:44.458640 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:44.458649 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.460172 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.460175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:44.461618 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:44.461629 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.464463 (+ 2834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.464466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:44.483119 (+ 18653us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:44.483158 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 05:11:44.483257 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 05:11:44.483368 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5083,"mutex_wait_us":29,"thread_start_us":119,"threads_started":1}
I20260807 05:11:44.485067 22057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:44.451494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35740 (local address 127.8.16.65:39619)
0807 05:11:44.457674 (+ 6180us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:44.457681 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:44.457710 (+ 29us) server_negotiation.cc:407] Connection header received
0807 05:11:44.457792 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:44.457798 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:44.458039 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:44.458224 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:11:44.458792 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.460027 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.463234 (+ 3207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.464106 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.464217 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:44.464394 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:44.483787 (+ 19393us) server_negotiation.cc:299] Negotiation successful
0807 05:11:44.483927 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6050,"thread_start_us":120,"threads_started":1}
I20260807 05:11:44.488062 21646 tablet_service.cc:1467] Processing CreateTablet for tablet bcb13c736b7343b78a8d2c125d056813 (DEFAULT_TABLE table=test-table [id=f2e0b86714944e6ba5ae04aad9e436e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:11:44.489138 21646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcb13c736b7343b78a8d2c125d056813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:11:44.490115 22063 tablet_bootstrap.cc:492] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc: Bootstrap starting.
I20260807 05:11:44.493651 22063 tablet_bootstrap.cc:654] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc: Neither blocks nor log segments found. Creating new log.
I20260807 05:11:44.494622 22063 log.cc:824] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc: Log is configured to *not* fsync() on all Append() calls
I20260807 05:11:44.502601 22065 tablet_bootstrap.cc:492] T bcb13c736b7343b78a8d2c125d056813 P 6e19e54494b449b2ba81ea45bf5bf6b3: Bootstrap starting.
I20260807 05:11:44.503340 22066 tablet_bootstrap.cc:492] T bcb13c736b7343b78a8d2c125d056813 P e2835474566449cca400fb5945831651: Bootstrap starting.
I20260807 05:11:44.506551 22066 tablet_bootstrap.cc:654] T bcb13c736b7343b78a8d2c125d056813 P e2835474566449cca400fb5945831651: Neither blocks nor log segments found. Creating new log.
I20260807 05:11:44.507922 22065 tablet_bootstrap.cc:654] T bcb13c736b7343b78a8d2c125d056813 P 6e19e54494b449b2ba81ea45bf5bf6b3: Neither blocks nor log segments found. Creating new log.
I20260807 05:11:44.508831 22065 log.cc:824] T bcb13c736b7343b78a8d2c125d056813 P 6e19e54494b449b2ba81ea45bf5bf6b3: Log is configured to *not* fsync() on all Append() calls
I20260807 05:11:44.510233 22066 log.cc:824] T bcb13c736b7343b78a8d2c125d056813 P e2835474566449cca400fb5945831651: Log is configured to *not* fsync() on all Append() calls
I20260807 05:11:44.511832 22063 tablet_bootstrap.cc:492] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc: No bootstrap required, opened a new log
I20260807 05:11:44.512100 22063 ts_tablet_manager.cc:1397] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.005s
I20260807 05:11:44.516332 22063 raft_consensus.cc:348] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2835474566449cca400fb5945831651" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39619 } } peers { permanent_uuid: "475d75f5c58d426ebd308e381a49f4bc" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 34279 } } peers { permanent_uuid: "6e19e54494b449b2ba81ea45bf5bf6b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42125 } }
I20260807 05:11:44.517522 22063 raft_consensus.cc:374] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:11:44.517638 22063 raft_consensus.cc:729] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 475d75f5c58d426ebd308e381a49f4bc, State: Initialized, Role: FOLLOWER
I20260807 05:11:44.518219 22063 consensus_queue.cc:260] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2835474566449cca400fb5945831651" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39619 } } peers { permanent_uuid: "475d75f5c58d426ebd308e381a49f4bc" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 34279 } } peers { permanent_uuid: "6e19e54494b449b2ba81ea45bf5bf6b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42125 } }
I20260807 05:11:44.519383 22063 ts_tablet_manager.cc:1428] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260807 05:11:44.519783 21879 heartbeater.cc:503] Master 127.8.16.126:34391 was elected leader, sending a full tablet report...
I20260807 05:11:44.526355 22065 tablet_bootstrap.cc:492] T bcb13c736b7343b78a8d2c125d056813 P 6e19e54494b449b2ba81ea45bf5bf6b3: No bootstrap required, opened a new log
I20260807 05:11:44.526633 22065 ts_tablet_manager.cc:1397] T bcb13c736b7343b78a8d2c125d056813 P 6e19e54494b449b2ba81ea45bf5bf6b3: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.000s
I20260807 05:11:44.530659 22065 raft_consensus.cc:348] T bcb13c736b7343b78a8d2c125d056813 P 6e19e54494b449b2ba81ea45bf5bf6b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2835474566449cca400fb5945831651" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39619 } } peers { permanent_uuid: "475d75f5c58d426ebd308e381a49f4bc" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 34279 } } peers { permanent_uuid: "6e19e54494b449b2ba81ea45bf5bf6b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42125 } }
I20260807 05:11:44.533895 22066 tablet_bootstrap.cc:492] T bcb13c736b7343b78a8d2c125d056813 P e2835474566449cca400fb5945831651: No bootstrap required, opened a new log
I20260807 05:11:44.534220 22066 ts_tablet_manager.cc:1397] T bcb13c736b7343b78a8d2c125d056813 P e2835474566449cca400fb5945831651: Time spent bootstrapping tablet: real 0.031s user 0.004s sys 0.002s
I20260807 05:11:44.538304 22066 raft_consensus.cc:348] T bcb13c736b7343b78a8d2c125d056813 P e2835474566449cca400fb5945831651 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2835474566449cca400fb5945831651" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39619 } } peers { permanent_uuid: "475d75f5c58d426ebd308e381a49f4bc" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 34279 } } peers { permanent_uuid: "6e19e54494b449b2ba81ea45bf5bf6b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42125 } }
I20260807 05:11:44.538725 22065 raft_consensus.cc:374] T bcb13c736b7343b78a8d2c125d056813 P 6e19e54494b449b2ba81ea45bf5bf6b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:11:44.538828 22065 raft_consensus.cc:729] T bcb13c736b7343b78a8d2c125d056813 P 6e19e54494b449b2ba81ea45bf5bf6b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e19e54494b449b2ba81ea45bf5bf6b3, State: Initialized, Role: FOLLOWER
I20260807 05:11:44.541518 22066 raft_consensus.cc:374] T bcb13c736b7343b78a8d2c125d056813 P e2835474566449cca400fb5945831651 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:11:44.541697 22065 consensus_queue.cc:260] T bcb13c736b7343b78a8d2c125d056813 P 6e19e54494b449b2ba81ea45bf5bf6b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2835474566449cca400fb5945831651" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39619 } } peers { permanent_uuid: "475d75f5c58d426ebd308e381a49f4bc" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 34279 } } peers { permanent_uuid: "6e19e54494b449b2ba81ea45bf5bf6b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42125 } }
I20260807 05:11:44.541913 22066 raft_consensus.cc:729] T bcb13c736b7343b78a8d2c125d056813 P e2835474566449cca400fb5945831651 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2835474566449cca400fb5945831651, State: Initialized, Role: FOLLOWER
I20260807 05:11:44.543231 22065 ts_tablet_manager.cc:1428] T bcb13c736b7343b78a8d2c125d056813 P 6e19e54494b449b2ba81ea45bf5bf6b3: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20260807 05:11:44.543568 22066 consensus_queue.cc:260] T bcb13c736b7343b78a8d2c125d056813 P e2835474566449cca400fb5945831651 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2835474566449cca400fb5945831651" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39619 } } peers { permanent_uuid: "475d75f5c58d426ebd308e381a49f4bc" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 34279 } } peers { permanent_uuid: "6e19e54494b449b2ba81ea45bf5bf6b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42125 } }
I20260807 05:11:44.544535 22066 ts_tablet_manager.cc:1428] T bcb13c736b7343b78a8d2c125d056813 P e2835474566449cca400fb5945831651: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260807 05:11:44.545059 21733 heartbeater.cc:503] Master 127.8.16.126:34391 was elected leader, sending a full tablet report...
I20260807 05:11:44.545137 22027 heartbeater.cc:503] Master 127.8.16.126:34391 was elected leader, sending a full tablet report...
W20260807 05:11:44.593880 21880 tablet.cc:2369] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 05:11:44.598537 21734 tablet.cc:2369] T bcb13c736b7343b78a8d2c125d056813 P e2835474566449cca400fb5945831651: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 05:11:44.611202 22028 tablet.cc:2369] T bcb13c736b7343b78a8d2c125d056813 P 6e19e54494b449b2ba81ea45bf5bf6b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:11:44.677850 22069 raft_consensus.cc:482] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:11:44.678110 22069 raft_consensus.cc:504] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2835474566449cca400fb5945831651" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39619 } } peers { permanent_uuid: "475d75f5c58d426ebd308e381a49f4bc" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 34279 } } peers { permanent_uuid: "6e19e54494b449b2ba81ea45bf5bf6b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42125 } }
I20260807 05:11:44.679553 22069 leader_election.cc:288] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2835474566449cca400fb5945831651 (127.8.16.65:39619), 6e19e54494b449b2ba81ea45bf5bf6b3 (127.8.16.67:42125)
I20260807 05:11:44.684247 22057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:44.679981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:59753 (local address 127.8.16.65:39619)
0807 05:11:44.680171 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:44.680176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:44.680416 (+ 240us) server_negotiation.cc:407] Connection header received
0807 05:11:44.680630 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:44.680633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:44.680676 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:44.680737 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:11:44.681089 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.681893 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.683302 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.683711 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.683808 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:44.683871 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:44.684039 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 05:11:44.684115 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 05:11:44.684275 22081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:44.679769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:39619 (local address 127.8.16.66:59753)
0807 05:11:44.680297 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:44.680312 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:44.680442 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:44.680762 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:44.680764 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:44.680781 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:11:44.680963 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:44.680970 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.682004 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.682007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:44.683156 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:44.683163 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.683831 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.683834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:44.683929 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:44.683953 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 05:11:44.684023 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 05:11:44.684108 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":45,"thread_start_us":132,"threads_started":1}
I20260807 05:11:44.684702 22059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:44.679942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:45383 (local address 127.8.16.67:42125)
0807 05:11:44.680132 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:44.680136 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:44.680493 (+ 357us) server_negotiation.cc:407] Connection header received
0807 05:11:44.680597 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:44.680600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:44.680656 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:44.680755 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:11:44.681157 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.681705 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.683299 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.683987 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.684090 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:44.684156 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:44.684482 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 05:11:44.684566 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260807 05:11:44.685057 21685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcb13c736b7343b78a8d2c125d056813" candidate_uuid: "475d75f5c58d426ebd308e381a49f4bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2835474566449cca400fb5945831651" is_pre_election: true
I20260807 05:11:44.685149 22080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:44.679769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:42125 (local address 127.8.16.66:45383)
0807 05:11:44.680415 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:44.680427 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:44.680511 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:44.680860 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:44.680863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:44.680875 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:11:44.681019 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:44.681025 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.681812 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.681815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:44.683115 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:44.683128 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.684223 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.684227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:44.684335 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:44.684358 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:11:44.684912 (+ 554us) client_negotiation.cc:240] Negotiation successful
0807 05:11:44.684991 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"mutex_wait_us":277,"thread_start_us":104,"threads_started":1}
I20260807 05:11:44.685388 21685 raft_consensus.cc:2454] T bcb13c736b7343b78a8d2c125d056813 P e2835474566449cca400fb5945831651 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 475d75f5c58d426ebd308e381a49f4bc in term 0.
I20260807 05:11:44.685726 21976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcb13c736b7343b78a8d2c125d056813" candidate_uuid: "475d75f5c58d426ebd308e381a49f4bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e19e54494b449b2ba81ea45bf5bf6b3" is_pre_election: true
I20260807 05:11:44.685947 21755 leader_election.cc:302] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 475d75f5c58d426ebd308e381a49f4bc, e2835474566449cca400fb5945831651; no voters:
I20260807 05:11:44.686012 21976 raft_consensus.cc:2454] T bcb13c736b7343b78a8d2c125d056813 P 6e19e54494b449b2ba81ea45bf5bf6b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 475d75f5c58d426ebd308e381a49f4bc in term 0.
I20260807 05:11:44.686265 22069 raft_consensus.cc:2781] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:11:44.686349 22069 raft_consensus.cc:482] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:11:44.686393 22069 raft_consensus.cc:3037] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:11:44.687521 22069 raft_consensus.cc:504] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2835474566449cca400fb5945831651" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39619 } } peers { permanent_uuid: "475d75f5c58d426ebd308e381a49f4bc" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 34279 } } peers { permanent_uuid: "6e19e54494b449b2ba81ea45bf5bf6b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42125 } }
I20260807 05:11:44.687851 22069 leader_election.cc:288] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc [CANDIDATE]: Term 1 election: Requested vote from peers e2835474566449cca400fb5945831651 (127.8.16.65:39619), 6e19e54494b449b2ba81ea45bf5bf6b3 (127.8.16.67:42125)
I20260807 05:11:44.688187 21685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcb13c736b7343b78a8d2c125d056813" candidate_uuid: "475d75f5c58d426ebd308e381a49f4bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2835474566449cca400fb5945831651"
I20260807 05:11:44.688308 21685 raft_consensus.cc:3037] T bcb13c736b7343b78a8d2c125d056813 P e2835474566449cca400fb5945831651 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:11:44.688788 21976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcb13c736b7343b78a8d2c125d056813" candidate_uuid: "475d75f5c58d426ebd308e381a49f4bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e19e54494b449b2ba81ea45bf5bf6b3"
I20260807 05:11:44.688920 21976 raft_consensus.cc:3037] T bcb13c736b7343b78a8d2c125d056813 P 6e19e54494b449b2ba81ea45bf5bf6b3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:11:44.689416 21685 raft_consensus.cc:2454] T bcb13c736b7343b78a8d2c125d056813 P e2835474566449cca400fb5945831651 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 475d75f5c58d426ebd308e381a49f4bc in term 1.
I20260807 05:11:44.689750 21755 leader_election.cc:302] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 475d75f5c58d426ebd308e381a49f4bc, e2835474566449cca400fb5945831651; no voters:
I20260807 05:11:44.689942 22069 raft_consensus.cc:2781] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:11:44.690157 22069 raft_consensus.cc:686] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc [term 1 LEADER]: Becoming Leader. State: Replica: 475d75f5c58d426ebd308e381a49f4bc, State: Running, Role: LEADER
I20260807 05:11:44.690266 21976 raft_consensus.cc:2454] T bcb13c736b7343b78a8d2c125d056813 P 6e19e54494b449b2ba81ea45bf5bf6b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 475d75f5c58d426ebd308e381a49f4bc in term 1.
I20260807 05:11:44.690500 22069 consensus_queue.cc:237] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2835474566449cca400fb5945831651" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39619 } } peers { permanent_uuid: "475d75f5c58d426ebd308e381a49f4bc" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 34279 } } peers { permanent_uuid: "6e19e54494b449b2ba81ea45bf5bf6b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42125 } }
I20260807 05:11:44.693806 21524 catalog_manager.cc:5495] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 475d75f5c58d426ebd308e381a49f4bc (127.8.16.66). New cstate: current_term: 1 leader_uuid: "475d75f5c58d426ebd308e381a49f4bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2835474566449cca400fb5945831651" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "475d75f5c58d426ebd308e381a49f4bc" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 34279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e19e54494b449b2ba81ea45bf5bf6b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42125 } health_report { overall_health: UNKNOWN } } }
I20260807 05:11:44.754997 22061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:44.750813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40200 (local address 127.8.16.66:34279)
0807 05:11:44.750939 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:44.750944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:44.750961 (+ 17us) server_negotiation.cc:407] Connection header received
0807 05:11:44.751083 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:44.751086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:44.751156 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:44.751280 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:11:44.751504 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.752312 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.753245 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:44.753617 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:44.753717 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:44.754001 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:11:44.754551 (+ 550us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:11:44.754675 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:44.754741 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 05:11:44.754803 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 05:11:44.761389 21976 raft_consensus.cc:1261] T bcb13c736b7343b78a8d2c125d056813 P 6e19e54494b449b2ba81ea45bf5bf6b3 [term 1 FOLLOWER]: Refusing update from remote peer 475d75f5c58d426ebd308e381a49f4bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:11:44.761438 21685 raft_consensus.cc:1261] T bcb13c736b7343b78a8d2c125d056813 P e2835474566449cca400fb5945831651 [term 1 FOLLOWER]: Refusing update from remote peer 475d75f5c58d426ebd308e381a49f4bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:11:44.762138 22082 consensus_queue.cc:1035] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e19e54494b449b2ba81ea45bf5bf6b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 42125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:11:44.762311 22069 consensus_queue.cc:1035] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2835474566449cca400fb5945831651" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 39619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:11:44.771785 22085 mvcc.cc:205] Tried to move back new op lower bound from 7315781651495411712 to 7315781651215122432. Current Snapshot: MvccSnapshot[applied={T|T < 7315781651495411712}]
I20260807 05:11:44.772562 22088 mvcc.cc:205] Tried to move back new op lower bound from 7315781651495411712 to 7315781651215122432. Current Snapshot: MvccSnapshot[applied={T|T < 7315781651495411712}]
I20260807 05:11:44.773623 22086 mvcc.cc:205] Tried to move back new op lower bound from 7315781651495411712 to 7315781651215122432. Current Snapshot: MvccSnapshot[applied={T|T < 7315781651495411712}]
I20260807 05:11:44.782881 21523 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:11:44.783092 21523 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:11:44.785730 21523 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc93185fc657494f926432313e6c2530: Sending DeleteTablet for 3 replicas of tablet bcb13c736b7343b78a8d2c125d056813
I20260807 05:11:44.786521 21805 tablet_service.cc:1514] Processing DeleteTablet for tablet bcb13c736b7343b78a8d2c125d056813 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:11:44 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:40192
I20260807 05:11:44.786536 21646 tablet_service.cc:1514] Processing DeleteTablet for tablet bcb13c736b7343b78a8d2c125d056813 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:11:44 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:35740
I20260807 05:11:44.786669 21933 tablet_service.cc:1514] Processing DeleteTablet for tablet bcb13c736b7343b78a8d2c125d056813 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:11:44 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:45124
I20260807 05:11:44.787092 22094 tablet_replica.cc:330] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc: stopping tablet replica
I20260807 05:11:44.787439 22094 raft_consensus.cc:2229] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:11:44.787519 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 21565
I20260807 05:11:44.789892 22094 raft_consensus.cc:2258] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:11:44.791507 22094 ts_tablet_manager.cc:1905] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:11:44.793653 22097 tablet_replica.cc:330] T bcb13c736b7343b78a8d2c125d056813 P 6e19e54494b449b2ba81ea45bf5bf6b3: stopping tablet replica
I20260807 05:11:44.794000 22097 raft_consensus.cc:2229] T bcb13c736b7343b78a8d2c125d056813 P 6e19e54494b449b2ba81ea45bf5bf6b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:11:44.794258 22094 ts_tablet_manager.cc:1918] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:11:44.794353 22097 raft_consensus.cc:2258] T bcb13c736b7343b78a8d2c125d056813 P 6e19e54494b449b2ba81ea45bf5bf6b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:11:44.795780 22097 ts_tablet_manager.cc:1905] T bcb13c736b7343b78a8d2c125d056813 P 6e19e54494b449b2ba81ea45bf5bf6b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:11:44.798384 22097 ts_tablet_manager.cc:1918] T bcb13c736b7343b78a8d2c125d056813 P 6e19e54494b449b2ba81ea45bf5bf6b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:11:44.798530 22097 log.cc:1196] T bcb13c736b7343b78a8d2c125d056813 P 6e19e54494b449b2ba81ea45bf5bf6b3: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/bcb13c736b7343b78a8d2c125d056813
I20260807 05:11:44.798887 22097 ts_tablet_manager.cc:1939] T bcb13c736b7343b78a8d2c125d056813 P 6e19e54494b449b2ba81ea45bf5bf6b3: Deleting consensus metadata
I20260807 05:11:44.799384 22094 log.cc:1196] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079354071073-8257-0/minicluster-data/ts-1/wal/wals/bcb13c736b7343b78a8d2c125d056813
I20260807 05:11:44.799775 22094 ts_tablet_manager.cc:1939] T bcb13c736b7343b78a8d2c125d056813 P 475d75f5c58d426ebd308e381a49f4bc: Deleting consensus metadata
I20260807 05:11:44.800105 21509 catalog_manager.cc:4841] TS 6e19e54494b449b2ba81ea45bf5bf6b3 (127.8.16.67:42125): tablet bcb13c736b7343b78a8d2c125d056813 (table test-table [id=f2e0b86714944e6ba5ae04aad9e436e5]) successfully deleted
W20260807 05:11:44.801040 21510 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.16.65:39619 (error 108)
I20260807 05:11:44.801517 21510 catalog_manager.cc:4841] TS 475d75f5c58d426ebd308e381a49f4bc (127.8.16.66:34279): tablet bcb13c736b7343b78a8d2c125d056813 (table test-table [id=f2e0b86714944e6ba5ae04aad9e436e5]) successfully deleted
I20260807 05:11:44.802058 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 21736
I20260807 05:11:44.802134 22060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:44.801842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:39619 (local address 127.0.0.1:35746)
0807 05:11:44.801973 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:44.802040 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.65:39619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 05:11:44.808097 21510 catalog_manager.cc:4584] TS e2835474566449cca400fb5945831651 (127.8.16.65:39619): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bcb13c736b7343b78a8d2c125d056813: Network error: Client connection negotiation failed: client connection to 127.8.16.65:39619: connect: Connection refused (error 111)
I20260807 05:11:44.808559 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 21882
I20260807 05:11:44.814970 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 21492
I20260807 05:11:44.821386 8257 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 05:11:45.166218 8257 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 05:11:45.166322 8257 mini_ranger.cc:66] Stopping Ranger...
I20260807 05:11:45.501470 8257 mini_ranger.cc:68] Stopped Ranger
2026-08-07 05:11:45.501 UTC [20499] LOG: received smart shutdown request
2026-08-07 05:11:45.505 UTC [20499] LOG: background worker "logical replication launcher" (PID 20508) exited with exit code 1
2026-08-07 05:11:45.505 UTC [20503] LOG: shutting down
2026-08-07 05:11:45.633 UTC [20499] LOG: database system is shut down
2026-08-07T05:11:45Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48424 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:11:45 dist-test-slave-49h3 krb5kdc[22108](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:11:45 dist-test-slave-49h3 krb5kdc[22108](info): set up 2 sockets
Aug 07 05:11:45 dist-test-slave-49h3 krb5kdc[22108](info): commencing operation
krb5kdc: starting...
W20260807 05:11:46.725013 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.966s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:11:46 dist-test-slave-49h3 krb5kdc[22108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:11:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:11:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:11:46.842276 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:38149
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:38831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:38149,127.8.16.125:36121,127.8.16.124:42367
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:11:46.950716 22122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:46.950968 22122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:46.951009 22122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:46.954934 22122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:11:46.954991 22122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:46.955009 22122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:46.955024 22122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:11:46.955040 22122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:11:46.960144 22122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:38831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:38149,127.8.16.125:36121,127.8.16.124:42367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:38149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:46.961143 22122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:46.962149 22122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:11:46.970177 22130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:46.970180 22128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:46.970345 22122 server_base.cc:1034] running on GCE node
W20260807 05:11:46.970188 22127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:46.971028 22122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:46.972103 22122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:46.973292 22122 hybrid_clock.cc:648] HybridClock initialized: now 1786079506973266 us; error 42 us; skew 500 ppm
Aug 07 05:11:46 dist-test-slave-49h3 krb5kdc[22108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079506, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:46.975986 22122 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:11:46.976904 22122 webserver.cc:460] Webserver started at http://127.8.16.126:43025/ using document root <none> and password file <none>
I20260807 05:11:46.977468 22122 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:46.977550 22122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:11:46.977757 22122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:11:46.979775 22122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "42476583a6654087a213a94098314633"
format_stamp: "Formatted at 2026-08-07 05:11:46 on dist-test-slave-49h3"
server_key: "f3ad9d1ec51f59bc8cbe30af4ae43fbb"
server_key_iv: "51211301ed427c49162cd6b28e18a6db"
server_key_version: "encryptionkey@0"
I20260807 05:11:46.980319 22122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "42476583a6654087a213a94098314633"
format_stamp: "Formatted at 2026-08-07 05:11:46 on dist-test-slave-49h3"
server_key: "f3ad9d1ec51f59bc8cbe30af4ae43fbb"
server_key_iv: "51211301ed427c49162cd6b28e18a6db"
server_key_version: "encryptionkey@0"
I20260807 05:11:46.983939 22122 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:11:46.986282 22137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:46.987445 22122 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:11:46.987560 22122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "42476583a6654087a213a94098314633"
format_stamp: "Formatted at 2026-08-07 05:11:46 on dist-test-slave-49h3"
server_key: "f3ad9d1ec51f59bc8cbe30af4ae43fbb"
server_key_iv: "51211301ed427c49162cd6b28e18a6db"
server_key_version: "encryptionkey@0"
I20260807 05:11:46.987656 22122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:47.005178 22122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:47.008169 22122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:47.008364 22122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:47.015370 22122 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:38149
I20260807 05:11:47.015388 22200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:38149 every 8 connection(s)
I20260807 05:11:47.016319 22122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:11:47.018249 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 22122
I20260807 05:11:47.018388 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:11:47.018671 8257 external_mini_cluster.cc:1428] Setting key d987b734ef357396a6941a8560ce1591
I20260807 05:11:47.019266 22201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:11:47.023279 22201 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38149 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.16.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.125@KRBTEST.COM" created.
Aug 07 05:11:47 dist-test-slave-49h3 krb5kdc[22108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079506, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:11:47.037621 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.025671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:38149 (local address 127.0.0.1:32834)
0807 05:11:47.026213 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.026256 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:47.027108 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:47.027752 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:47.027757 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:47.028160 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:47.028412 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:47.028425 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.029680 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.029683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:47.030402 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:47.030411 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.031021 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.031025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:47.031120 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.031692 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:47.031708 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:47.033088 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:47.034853 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:47.034861 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:47.034871 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:47.035080 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:47.035372 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:47.035375 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:47.035377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:47.035481 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:47.036010 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:47.036016 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:47.036145 (+ 129us) client_negotiation.cc:769] Sending connection context
0807 05:11:47.036291 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 05:11:47.036655 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":85,"thread_start_us":158,"threads_started":1}
I20260807 05:11:47.037621 22205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.026674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32834 (local address 127.8.16.126:38149)
0807 05:11:47.027052 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:47.027067 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:47.027112 (+ 45us) server_negotiation.cc:407] Connection header received
0807 05:11:47.027400 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:47.027405 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:47.027575 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:47.027743 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:47.028547 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.029571 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.030511 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.030882 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.031024 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.033224 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:47.033229 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:47.033239 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:47.033259 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:47.034757 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.035183 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.035188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.035189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.035289 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.035561 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.035564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.035565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.035908 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:47.036024 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:47.036363 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 05:11:47.036572 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":94,"threads_started":1}
I20260807 05:11:47.040439 22201 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:11:47.041148 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.040894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:36121 (local address 127.0.0.1:40486)
0807 05:11:47.041016 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.041087 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.125:36121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 05:11:47.041342 22138 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.16.125:36121: connect: Connection refused (error 111)
I20260807 05:11:47.042181 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.042002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:36121 (local address 127.0.0.1:40490)
0807 05:11:47.042083 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.042126 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.125:36121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 05:11:47.042320 22201 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.125:36121: Network error: Client connection negotiation failed: client connection to 127.8.16.125:36121: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.125.keytab.
Entry for principal kudu/127.8.16.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.125.keytab.
I20260807 05:11:47.043867 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.125:36121
--webserver_interface=127.8.16.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.125.keytab
--principal=kudu/127.8.16.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:38831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:38149,127.8.16.125:36121,127.8.16.124:42367
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:11:47.091598 22201 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } attempt: 1
I20260807 05:11:47.092482 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.092120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:36121 (local address 127.0.0.1:40504)
0807 05:11:47.092307 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.092386 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.125:36121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 05:11:47.093202 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.092957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:36121 (local address 127.0.0.1:40510)
0807 05:11:47.093057 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.093116 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.125:36121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 05:11:47.093382 22201 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.125:36121: Network error: Client connection negotiation failed: client connection to 127.8.16.125:36121: connect: Connection refused (error 111)
W20260807 05:11:47.151041 22208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:47.151284 22208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:47.151320 22208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:47.155236 22208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:11:47.155292 22208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:47.155309 22208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:47.155324 22208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:11:47.155339 22208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:11:47.160355 22208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:38831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:38149,127.8.16.125:36121,127.8.16.124:42367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.125.keytab
--principal=kudu/127.8.16.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.125:36121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.16.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:47.161331 22208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:47.161594 22201 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } attempt: 2
I20260807 05:11:47.162395 22208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 05:11:47.162437 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.162107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:36121 (local address 127.0.0.1:40514)
0807 05:11:47.162278 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.162350 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.125:36121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 05:11:47.163163 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.162935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:36121 (local address 127.0.0.1:40522)
0807 05:11:47.163039 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.163092 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.125:36121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 05:11:47.163317 22201 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.125:36121: Network error: Client connection negotiation failed: client connection to 127.8.16.125:36121: connect: Connection refused (error 111)
W20260807 05:11:47.170321 22213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:47.170301 22216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:47.170301 22214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:47.170708 22208 server_base.cc:1034] running on GCE node
I20260807 05:11:47.171211 22208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:47.172241 22208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:47.173408 22208 hybrid_clock.cc:648] HybridClock initialized: now 1786079507173392 us; error 33 us; skew 500 ppm
Aug 07 05:11:47 dist-test-slave-49h3 krb5kdc[22108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079507, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:47.175993 22208 init.cc:377] Logged in from keytab as kudu/127.8.16.125@KRBTEST.COM (short username kudu)
I20260807 05:11:47.176828 22208 webserver.cc:460] Webserver started at http://127.8.16.125:44499/ using document root <none> and password file <none>
I20260807 05:11:47.177349 22208 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:47.177414 22208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:11:47.177626 22208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:11:47.179626 22208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-1/data/instance:
uuid: "3f1cd654dc974ed58c18bb041b6f162e"
format_stamp: "Formatted at 2026-08-07 05:11:47 on dist-test-slave-49h3"
server_key: "d6849797e6abdab6ad9e714e03cd5a6d"
server_key_iv: "9f13cb824c40e6df4a58ffc255066d29"
server_key_version: "encryptionkey@0"
I20260807 05:11:47.180168 22208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-1/wal/instance:
uuid: "3f1cd654dc974ed58c18bb041b6f162e"
format_stamp: "Formatted at 2026-08-07 05:11:47 on dist-test-slave-49h3"
server_key: "d6849797e6abdab6ad9e714e03cd5a6d"
server_key_iv: "9f13cb824c40e6df4a58ffc255066d29"
server_key_version: "encryptionkey@0"
I20260807 05:11:47.183693 22208 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:11:47.186106 22225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:47.187270 22208 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:11:47.187386 22208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-1/wal
uuid: "3f1cd654dc974ed58c18bb041b6f162e"
format_stamp: "Formatted at 2026-08-07 05:11:47 on dist-test-slave-49h3"
server_key: "d6849797e6abdab6ad9e714e03cd5a6d"
server_key_iv: "9f13cb824c40e6df4a58ffc255066d29"
server_key_version: "encryptionkey@0"
I20260807 05:11:47.187479 22208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:47.198812 22208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:47.201574 22208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:47.201748 22208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:47.208253 22208 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.125:36121
I20260807 05:11:47.208264 22281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.125:36121 every 8 connection(s)
I20260807 05:11:47.209124 22208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-1/data/info.pb
I20260807 05:11:47.210217 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 22208
I20260807 05:11:47.210336 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-1/wal/instance
I20260807 05:11:47.210593 8257 external_mini_cluster.cc:1428] Setting key fcaebdbdcc81f09c87b45b6429e77047
I20260807 05:11:47.211822 22282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:11:47.215687 22282 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38149 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.16.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.124@KRBTEST.COM" created.
Aug 07 05:11:47 dist-test-slave-49h3 krb5kdc[22108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079507, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.125@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:11:47.228972 22205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.217902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32850 (local address 127.8.16.126:38149)
0807 05:11:47.218055 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:47.218060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:47.218939 (+ 879us) server_negotiation.cc:407] Connection header received
0807 05:11:47.219848 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:47.219851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:47.219898 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:47.219971 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:47.221051 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.221669 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.222794 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.223135 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.223243 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.225603 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:47.225607 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:47.225610 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:47.225637 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:47.226973 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.227460 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.227463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.227465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.227541 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.227846 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.227850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.227851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.228034 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:47.228133 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:47.228659 (+ 526us) server_negotiation.cc:299] Negotiation successful
0807 05:11:47.228799 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 05:11:47.229825 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.218258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:38149 (local address 127.0.0.1:32850)
0807 05:11:47.218769 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.218826 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:47.219671 (+ 845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:47.220104 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:47.220110 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:47.220418 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:47.220937 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:47.220950 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.221796 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.221799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:47.222662 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:47.222671 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.223292 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.223297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:47.223457 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.224029 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:47.224045 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:47.225420 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:47.227084 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:47.227090 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:47.227106 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:47.227368 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:47.227621 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:47.227624 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:47.227626 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:47.227763 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:47.228146 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:47.228152 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:47.228417 (+ 265us) client_negotiation.cc:769] Sending connection context
0807 05:11:47.228587 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 05:11:47.228913 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":114,"threads_started":1}
I20260807 05:11:47.231208 22282 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.124.keytab.
Entry for principal kudu/127.8.16.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.124.keytab.
I20260807 05:11:47.236145 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.124:42367
--webserver_interface=127.8.16.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.124.keytab
--principal=kudu/127.8.16.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:38831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:38149,127.8.16.125:36121,127.8.16.124:42367
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 05:11:47 dist-test-slave-49h3 krb5kdc[22108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079507, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.125@KRBTEST.COM for kudu/127.8.16.125@KRBTEST.COM
I20260807 05:11:47.239705 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.231702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:36121 (local address 127.0.0.1:40534)
0807 05:11:47.231825 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.231838 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:47.231915 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:47.232436 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:47.232438 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:47.232607 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:47.232731 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:47.232736 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.233755 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.233758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:47.234337 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:47.234346 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.234962 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.234965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:47.235047 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.235450 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:47.235461 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:47.236557 (+ 1096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:47.238313 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:47.238316 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:47.238318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:47.238549 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:47.238787 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:47.238790 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:47.238792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:47.238883 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:47.239353 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:47.239355 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:47.239428 (+ 73us) client_negotiation.cc:769] Sending connection context
0807 05:11:47.239499 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 05:11:47.239591 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 05:11:47.240028 22287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.231796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40534 (local address 127.8.16.125:36121)
0807 05:11:47.232122 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:47.232130 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:47.232149 (+ 19us) server_negotiation.cc:407] Connection header received
0807 05:11:47.232181 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:47.232185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:47.232330 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:47.232429 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:47.232817 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.233660 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.234440 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.234826 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.234951 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.236683 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:47.236694 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:47.236708 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:47.236732 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:47.238211 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.238642 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.238646 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.238648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.238713 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.238959 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.238961 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.238963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.239260 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:47.239385 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:47.239593 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 05:11:47.239730 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":128,"threads_started":1}
I20260807 05:11:47.241743 22282 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:11:47.242413 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.242190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:42367 (local address 127.0.0.1:56868)
0807 05:11:47.242295 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.242360 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:42367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 05:11:47.242604 22229 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.16.124:42367: connect: Connection refused (error 111)
I20260807 05:11:47.243361 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.243183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:42367 (local address 127.0.0.1:56878)
0807 05:11:47.243272 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.243316 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:42367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 05:11:47.243477 22282 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.124:42367: Network error: Client connection negotiation failed: client connection to 127.8.16.124:42367: connect: Connection refused (error 111)
I20260807 05:11:47.254501 22201 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } attempt: 3
Aug 07 05:11:47 dist-test-slave-49h3 krb5kdc[22108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079506, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for kudu/127.8.16.125@KRBTEST.COM
I20260807 05:11:47.263309 22287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.255100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40542 (local address 127.8.16.125:36121)
0807 05:11:47.255223 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:47.255227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:47.255246 (+ 19us) server_negotiation.cc:407] Connection header received
0807 05:11:47.255391 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:47.255395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:47.255438 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:47.255501 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:47.256329 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.257260 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.258092 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.258334 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.258426 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.260441 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:47.260444 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:47.260447 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:47.260474 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:47.261821 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.262208 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.262212 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.262214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.262289 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.262528 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.262531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.262533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.262713 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:47.262796 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:47.263044 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 05:11:47.263153 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 05:11:47.263309 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.255030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:36121 (local address 127.0.0.1:40542)
0807 05:11:47.255186 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.255201 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:47.255303 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:47.255533 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:47.255535 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:47.255711 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:47.256207 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:47.256216 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.257370 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.257373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:47.257983 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:47.257994 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.258437 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.258441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:47.258549 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.259038 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:47.259050 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:47.260316 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:47.261919 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:47.261923 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:47.261925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:47.262123 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:47.262368 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:47.262371 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:47.262373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:47.262451 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:47.262802 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:47.262805 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:47.262899 (+ 94us) client_negotiation.cc:769] Sending connection context
0807 05:11:47.262986 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 05:11:47.263141 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 05:11:47.264062 22201 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:11:47.264734 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.264511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:42367 (local address 127.0.0.1:56884)
0807 05:11:47.264618 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.264670 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:42367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 05:11:47.265390 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.265160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:42367 (local address 127.0.0.1:56886)
0807 05:11:47.265292 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.265336 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:42367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260807 05:11:47.265515 22201 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.124:42367: Network error: Client connection negotiation failed: client connection to 127.8.16.124:42367: connect: Connection refused (error 111)
I20260807 05:11:47.292690 22282 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } attempt: 1
I20260807 05:11:47.293560 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.293240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:42367 (local address 127.0.0.1:56896)
0807 05:11:47.293391 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.293473 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:42367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 05:11:47.294224 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.294018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:42367 (local address 127.0.0.1:56910)
0807 05:11:47.294118 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.294167 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:42367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 05:11:47.294340 22282 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.124:42367: Network error: Client connection negotiation failed: client connection to 127.8.16.124:42367: connect: Connection refused (error 111)
I20260807 05:11:47.296659 22201 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } attempt: 1
I20260807 05:11:47.297293 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.297022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:42367 (local address 127.0.0.1:56918)
0807 05:11:47.297167 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.297222 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:42367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 05:11:47.297950 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.297764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:42367 (local address 127.0.0.1:56934)
0807 05:11:47.297856 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.297897 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:42367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 05:11:47.298054 22201 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.124:42367: Network error: Client connection negotiation failed: client connection to 127.8.16.124:42367: connect: Connection refused (error 111)
W20260807 05:11:47.345952 22288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:47.346228 22288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:47.346287 22288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:47.350224 22288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:11:47.350288 22288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:47.350315 22288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:47.350335 22288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:11:47.350356 22288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:11:47.355588 22288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:38831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:38149,127.8.16.125:36121,127.8.16.124:42367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.124.keytab
--principal=kudu/127.8.16.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.124:42367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.16.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:47.356629 22288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:47.357645 22288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 05:11:47.362483 22282 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } attempt: 2
I20260807 05:11:47.363301 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.362988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:42367 (local address 127.0.0.1:56942)
0807 05:11:47.363156 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.363223 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:42367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 05:11:47.363998 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.363799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:42367 (local address 127.0.0.1:56954)
0807 05:11:47.363897 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.363944 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:42367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 05:11:47.364110 22282 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.124:42367: Network error: Client connection negotiation failed: client connection to 127.8.16.124:42367: connect: Connection refused (error 111)
W20260807 05:11:47.365927 22294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:47.366027 22288 server_base.cc:1034] running on GCE node
W20260807 05:11:47.365917 22297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:47.365917 22295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:47.366689 22288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:47.367815 22288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:47.368964 22288 hybrid_clock.cc:648] HybridClock initialized: now 1786079507368948 us; error 27 us; skew 500 ppm
Aug 07 05:11:47 dist-test-slave-49h3 krb5kdc[22108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079507, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:47.371632 22288 init.cc:377] Logged in from keytab as kudu/127.8.16.124@KRBTEST.COM (short username kudu)
I20260807 05:11:47.372512 22288 webserver.cc:460] Webserver started at http://127.8.16.124:40447/ using document root <none> and password file <none>
I20260807 05:11:47.373015 22288 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:47.373075 22288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:11:47.373189 22201 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } attempt: 2
I20260807 05:11:47.373267 22288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:11:47.373900 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.373646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:42367 (local address 127.0.0.1:56962)
0807 05:11:47.373775 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.373826 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:42367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 05:11:47.374547 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.374348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:42367 (local address 127.0.0.1:56970)
0807 05:11:47.374452 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.374494 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:42367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 05:11:47.374675 22201 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.124:42367: Network error: Client connection negotiation failed: client connection to 127.8.16.124:42367: connect: Connection refused (error 111)
I20260807 05:11:47.375474 22288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-2/data/instance:
uuid: "97475af900cc4c8d8d7390851c6e297b"
format_stamp: "Formatted at 2026-08-07 05:11:47 on dist-test-slave-49h3"
server_key: "ba88d3cfa0d5615cad635891f6bc5586"
server_key_iv: "5c6b77780a48c4d0bf37db9afcc41378"
server_key_version: "encryptionkey@0"
I20260807 05:11:47.375995 22288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-2/wal/instance:
uuid: "97475af900cc4c8d8d7390851c6e297b"
format_stamp: "Formatted at 2026-08-07 05:11:47 on dist-test-slave-49h3"
server_key: "ba88d3cfa0d5615cad635891f6bc5586"
server_key_iv: "5c6b77780a48c4d0bf37db9afcc41378"
server_key_version: "encryptionkey@0"
I20260807 05:11:47.379560 22288 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:11:47.381754 22304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:47.382885 22288 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:11:47.382992 22288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-2/wal
uuid: "97475af900cc4c8d8d7390851c6e297b"
format_stamp: "Formatted at 2026-08-07 05:11:47 on dist-test-slave-49h3"
server_key: "ba88d3cfa0d5615cad635891f6bc5586"
server_key_iv: "5c6b77780a48c4d0bf37db9afcc41378"
server_key_version: "encryptionkey@0"
I20260807 05:11:47.383078 22288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:47.397377 22288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:47.400484 22288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:47.400677 22288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:47.407384 22288 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.124:42367
I20260807 05:11:47.407400 22359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.124:42367 every 8 connection(s)
I20260807 05:11:47.408286 22288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-2/data/info.pb
I20260807 05:11:47.411000 22360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:11:47.413060 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 22288
I20260807 05:11:47.413182 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079354071073-8257-0/minicluster-data/master-2/wal/instance
I20260807 05:11:47.413436 8257 external_mini_cluster.cc:1428] Setting key 90a2f9e58aff4b76874972bbdc967fac
I20260807 05:11:47.414902 22360 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38149 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 05:11:47 dist-test-slave-49h3 krb5kdc[22108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079507, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.124@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
Aug 07 05:11:47 dist-test-slave-49h3 krb5kdc[22108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.124@KRBTEST.COM
I20260807 05:11:47.426311 22205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.416588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32858 (local address 127.8.16.126:38149)
0807 05:11:47.416800 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:47.416805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:47.417174 (+ 369us) server_negotiation.cc:407] Connection header received
0807 05:11:47.417292 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:47.417296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:47.417347 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:47.417407 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:47.418231 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.418730 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.419765 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.420025 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.420133 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.422402 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:47.422406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:47.422409 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:47.422440 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:47.424076 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.424611 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.424614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.424616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.424705 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.425110 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.425114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.425116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.425347 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:47.425468 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:47.426005 (+ 537us) server_negotiation.cc:299] Negotiation successful
0807 05:11:47.426145 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260807 05:11:47.427156 22364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.416477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:38149 (local address 127.0.0.1:32858)
0807 05:11:47.417024 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.417071 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:47.417200 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:47.417498 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:47.417503 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:47.417918 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:47.418111 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:47.418124 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.418838 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.418840 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:47.419639 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:47.419647 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.420164 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.420167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:47.420248 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.420760 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:47.420775 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:47.422202 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:47.424179 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:47.424187 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:47.424195 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:47.424511 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:47.424802 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:47.424805 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:47.424808 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:47.425000 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:47.425473 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:47.425480 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:47.425776 (+ 296us) client_negotiation.cc:769] Sending connection context
0807 05:11:47.425947 (+ 171us) client_negotiation.cc:240] Negotiation successful
0807 05:11:47.426298 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":40,"thread_start_us":201,"threads_started":1}
I20260807 05:11:47.427160 22363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.414851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56982 (local address 127.8.16.124:42367)
0807 05:11:47.415239 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:47.415257 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:47.415300 (+ 43us) server_negotiation.cc:407] Connection header received
0807 05:11:47.415927 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:47.415943 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:47.416261 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:47.416543 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:47.417349 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.418600 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.419425 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.419880 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.419987 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.422785 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:47.422792 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:47.422804 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:47.422829 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:47.424523 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.425012 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.425017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.425018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.425112 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.425380 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.425383 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.425385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.425754 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:47.425891 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:47.426157 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 05:11:47.426354 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":251,"spinlock_wait_cycles":2944,"thread_start_us":115,"threads_started":1}
I20260807 05:11:47.428311 22360 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 05:11:47 dist-test-slave-49h3 krb5kdc[22108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079507, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.124@KRBTEST.COM for kudu/127.8.16.125@KRBTEST.COM
Aug 07 05:11:47 dist-test-slave-49h3 krb5kdc[22108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:11:47.437090 22364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.428741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:36121 (local address 127.0.0.1:40550)
0807 05:11:47.429077 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.429088 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:47.429178 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:47.429574 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:47.429577 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:47.429760 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:47.429885 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:47.429892 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.430790 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.430792 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:47.431499 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:47.431509 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.431946 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.431949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:47.432022 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.432497 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:47.432513 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:47.433960 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:47.435667 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:47.435670 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:47.435673 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:47.435889 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:47.436166 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:47.436170 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:47.436172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:47.436260 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:47.436608 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:47.436611 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:47.436710 (+ 99us) client_negotiation.cc:769] Sending connection context
0807 05:11:47.436822 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 05:11:47.436937 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281}
I20260807 05:11:47.437096 22287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.428876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40550 (local address 127.8.16.125:36121)
0807 05:11:47.429002 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:47.429006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:47.429154 (+ 148us) server_negotiation.cc:407] Connection header received
0807 05:11:47.429291 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:47.429295 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:47.429349 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:47.429401 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:47.429991 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.430663 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.431611 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.431847 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.431943 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.434098 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:47.434102 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:47.434106 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:47.434134 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:47.435561 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.435990 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.435993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.435994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.436067 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.436348 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.436351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.436353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.436519 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:47.436592 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:47.436855 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 05:11:47.436944 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 05:11:47.438086 22360 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:11:47.438433 22205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.429645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32870 (local address 127.8.16.126:38149)
0807 05:11:47.430293 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:47.430296 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:47.430306 (+ 10us) server_negotiation.cc:407] Connection header received
0807 05:11:47.430334 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:47.430336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:47.430367 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:47.430425 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:47.431080 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.431744 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.432731 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.433056 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.433148 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.435507 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:47.435510 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:47.435512 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:47.435535 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:47.436900 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.437328 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.437332 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.437334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.437407 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.437668 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.437671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.437673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.437855 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:47.438078 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:47.438180 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 05:11:47.438287 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579}
Aug 07 05:11:47 dist-test-slave-49h3 krb5kdc[22108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079507, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.124@KRBTEST.COM for kudu/127.8.16.124@KRBTEST.COM
Aug 07 05:11:47 dist-test-slave-49h3 krb5kdc[22108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.125@KRBTEST.COM
I20260807 05:11:47.445418 22364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.438527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:42367 (local address 127.0.0.1:56990)
0807 05:11:47.438610 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.438622 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:47.438688 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:47.439000 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:47.439002 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:47.439146 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:47.439205 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:47.439210 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.440027 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.440030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:47.440519 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:47.440526 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.440928 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.440931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:47.441025 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.441438 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:47.441471 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:47.442757 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:47.444149 (+ 1392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:47.444151 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:47.444153 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:47.444323 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:47.444551 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:47.444554 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:47.444556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:47.444624 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:47.444931 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:47.444933 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:47.445010 (+ 77us) client_negotiation.cc:769] Sending connection context
0807 05:11:47.445091 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 05:11:47.445218 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":17}
I20260807 05:11:47.445923 22363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.438670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56990 (local address 127.8.16.124:42367)
0807 05:11:47.438852 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:47.438855 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:47.438864 (+ 9us) server_negotiation.cc:407] Connection header received
0807 05:11:47.438887 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:47.438889 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:47.438917 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:47.438975 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:47.439298 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.439938 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.440615 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.440839 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.440919 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.442879 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:47.442882 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:47.442884 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:47.442904 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:47.444059 (+ 1155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.444408 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.444411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.444413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.444478 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.444695 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.444698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.444700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.444851 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:47.444930 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:47.445140 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 05:11:47.445275 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":129}
I20260807 05:11:47.447048 22287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.439408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40554 (local address 127.8.16.125:36121)
0807 05:11:47.439516 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:47.439520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:47.439533 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:11:47.439566 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:47.439568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:47.439606 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:47.439684 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:47.440167 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.440620 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.441351 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.441678 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.441779 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.444064 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:47.444068 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:47.444070 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:47.444093 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:47.445386 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.445900 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.445904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.445905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.445978 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.446302 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.446306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.446308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.446474 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:47.446706 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:47.446807 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 05:11:47.446909 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 05:11:47.449013 22360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b: Bootstrap starting.
I20260807 05:11:47.450239 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:11:47.451538 22360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b: Neither blocks nor log segments found. Creating new log.
I20260807 05:11:47.452354 22360 log.cc:824] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b: Log is configured to *not* fsync() on all Append() calls
I20260807 05:11:47.454905 22360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b: No bootstrap required, opened a new log
I20260807 05:11:47.455255 22282 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } attempt: 3
I20260807 05:11:47.459062 22360 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42476583a6654087a213a94098314633" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38149 } } peers { permanent_uuid: "3f1cd654dc974ed58c18bb041b6f162e" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } } peers { permanent_uuid: "97475af900cc4c8d8d7390851c6e297b" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } }
I20260807 05:11:47.459328 22360 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:11:47.459386 22360 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97475af900cc4c8d8d7390851c6e297b, State: Initialized, Role: FOLLOWER
I20260807 05:11:47.459890 22360 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42476583a6654087a213a94098314633" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38149 } } peers { permanent_uuid: "3f1cd654dc974ed58c18bb041b6f162e" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } } peers { permanent_uuid: "97475af900cc4c8d8d7390851c6e297b" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } }
I20260807 05:11:47.461347 22205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.452179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32884 (local address 127.8.16.126:38149)
0807 05:11:47.452297 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:47.452302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:47.452508 (+ 206us) server_negotiation.cc:407] Connection header received
0807 05:11:47.452696 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:47.452700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:47.452752 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:47.452806 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:47.453753 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.454463 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.455529 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.455841 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.455920 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.457635 (+ 1715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:47.457638 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:47.457641 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:47.457669 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:47.459191 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.459763 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.459767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.459769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.459855 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.460145 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.460148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.460150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.460376 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:47.460489 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:47.461067 (+ 578us) server_negotiation.cc:299] Negotiation successful
0807 05:11:47.461182 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 05:11:47.461330 22360 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:11:47.462066 22287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.452462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40562 (local address 127.8.16.125:36121)
0807 05:11:47.452593 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:47.452597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:47.453071 (+ 474us) server_negotiation.cc:407] Connection header received
0807 05:11:47.453107 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:47.453109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:47.453150 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:47.453225 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:47.453997 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.454677 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.455584 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.455893 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.455959 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.457535 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:47.457539 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:47.457542 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:47.457567 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:47.459561 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.460257 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.460260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.460262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.460346 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.460636 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.460640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.460641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.461086 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:47.461212 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:47.461808 (+ 596us) server_negotiation.cc:299] Negotiation successful
0807 05:11:47.461912 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68,"spinlock_wait_cycles":768}
Aug 07 05:11:47 dist-test-slave-49h3 krb5kdc[22108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079507, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.125@KRBTEST.COM for kudu/127.8.16.124@KRBTEST.COM
I20260807 05:11:47.462728 22374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42476583a6654087a213a94098314633" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38149 } } peers { permanent_uuid: "3f1cd654dc974ed58c18bb041b6f162e" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } } peers { permanent_uuid: "97475af900cc4c8d8d7390851c6e297b" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } } }
I20260807 05:11:47.462872 22374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b [sys.catalog]: This master's current role is: FOLLOWER
W20260807 05:11:47.467362 22386 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:11:47.467494 22386 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:11:47.465597 22288 master_runner.cc:186] Error getting master registration for 127.8.16.126:38149: OK, instance_id { permanent_uuid: "42476583a6654087a213a94098314633" instance_seqno: 1786079507013352 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 05:11:47.469159 22288 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 05:11:47.470614 22373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.456017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57012 (local address 127.8.16.124:42367)
0807 05:11:47.456595 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:47.456604 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:47.456616 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:11:47.457536 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:47.457540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:47.457602 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:47.458146 (+ 544us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:47.458279 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.459055 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.460089 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.460409 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.460814 (+ 405us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.462763 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:47.462771 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:47.462774 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:47.462800 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:47.464916 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.469278 (+ 4362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.469282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.469284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.469392 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.469745 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.469748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.469750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.470021 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:47.470115 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:47.470395 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 05:11:47.470485 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":478,"spinlock_wait_cycles":2304,"thread_start_us":130,"threads_started":1}
I20260807 05:11:47.470638 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.455606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:42367 (local address 127.0.0.1:57012)
0807 05:11:47.455813 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.455826 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:47.455893 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:47.457752 (+ 1859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:47.457755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:47.457913 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:47.458009 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:47.458016 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.459181 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.459184 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:47.459948 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:47.459959 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.460524 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.460527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:47.460612 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.461029 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:47.461056 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:47.462465 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:47.465027 (+ 2562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:47.465030 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:47.465032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:47.465259 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:47.469563 (+ 4304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:47.469566 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:47.469569 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:47.469658 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:47.470119 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:47.470122 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:47.470233 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 05:11:47.470339 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 05:11:47.470474 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"spinlock_wait_cycles":8832}
I20260807 05:11:47.471419 22363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.452718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57006 (local address 127.8.16.124:42367)
0807 05:11:47.452817 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:47.452821 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:47.453043 (+ 222us) server_negotiation.cc:407] Connection header received
0807 05:11:47.453114 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:47.453116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:47.453154 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:47.453216 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:47.463935 (+ 10719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.464540 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.465399 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.465701 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.465788 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.467331 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:47.467334 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:47.467337 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:47.467360 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:47.468912 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.469373 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.469376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.469378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.469516 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.469797 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.469800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.469802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.470018 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:47.470151 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:47.471189 (+ 1038us) server_negotiation.cc:299] Negotiation successful
0807 05:11:47.471294 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":40}
I20260807 05:11:47.473273 22282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f1cd654dc974ed58c18bb041b6f162e: Bootstrap starting.
I20260807 05:11:47.473810 22201 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } attempt: 3
I20260807 05:11:47.476516 22282 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f1cd654dc974ed58c18bb041b6f162e: Neither blocks nor log segments found. Creating new log.
I20260807 05:11:47.477386 22282 log.cc:824] T 00000000000000000000000000000000 P 3f1cd654dc974ed58c18bb041b6f162e: Log is configured to *not* fsync() on all Append() calls
Aug 07 05:11:47 dist-test-slave-49h3 krb5kdc[22108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079506, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for kudu/127.8.16.124@KRBTEST.COM
I20260807 05:11:47.479843 22282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f1cd654dc974ed58c18bb041b6f162e: No bootstrap required, opened a new log
I20260807 05:11:47.482909 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.474471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:42367 (local address 127.0.0.1:57018)
0807 05:11:47.474724 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:47.474738 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:47.474816 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:47.475176 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:47.475180 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:47.475374 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:47.475483 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:47.475490 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.476362 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.476365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:47.477144 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:47.477157 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.478102 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.478105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:47.478199 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.478701 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:47.478715 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:47.479824 (+ 1109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:47.481365 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:47.481368 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:47.481370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:47.481634 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:47.481912 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:47.481915 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:47.481917 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:47.482012 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:47.482442 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:47.482445 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:47.482526 (+ 81us) client_negotiation.cc:769] Sending connection context
0807 05:11:47.482611 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 05:11:47.482747 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176}
I20260807 05:11:47.482909 22363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:47.474711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57018 (local address 127.8.16.124:42367)
0807 05:11:47.474979 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:47.474984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:47.474995 (+ 11us) server_negotiation.cc:407] Connection header received
0807 05:11:47.475023 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:47.475026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:47.475060 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:47.475138 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:47.475601 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.476241 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.477736 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:47.477998 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:47.478074 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:47.479945 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:47.479948 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:47.479951 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:47.479974 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:47.481259 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.481738 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.481741 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.481743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.481822 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:47.482112 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:47.482115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:47.482116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:47.482315 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:47.482399 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:47.482680 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 05:11:47.482773 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201}
I20260807 05:11:47.484745 22282 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f1cd654dc974ed58c18bb041b6f162e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42476583a6654087a213a94098314633" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38149 } } peers { permanent_uuid: "3f1cd654dc974ed58c18bb041b6f162e" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } } peers { permanent_uuid: "97475af900cc4c8d8d7390851c6e297b" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } }
I20260807 05:11:47.484957 22282 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f1cd654dc974ed58c18bb041b6f162e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:11:47.485005 22282 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f1cd654dc974ed58c18bb041b6f162e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1cd654dc974ed58c18bb041b6f162e, State: Initialized, Role: FOLLOWER
I20260807 05:11:47.485550 22201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633: Bootstrap starting.
I20260807 05:11:47.485504 22282 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f1cd654dc974ed58c18bb041b6f162e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42476583a6654087a213a94098314633" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38149 } } peers { permanent_uuid: "3f1cd654dc974ed58c18bb041b6f162e" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } } peers { permanent_uuid: "97475af900cc4c8d8d7390851c6e297b" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } }
I20260807 05:11:47.486177 22389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f1cd654dc974ed58c18bb041b6f162e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42476583a6654087a213a94098314633" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38149 } } peers { permanent_uuid: "3f1cd654dc974ed58c18bb041b6f162e" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } } peers { permanent_uuid: "97475af900cc4c8d8d7390851c6e297b" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } } }
I20260807 05:11:47.486364 22389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f1cd654dc974ed58c18bb041b6f162e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:11:47.487058 22282 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f1cd654dc974ed58c18bb041b6f162e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:11:47.488318 22201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633: Neither blocks nor log segments found. Creating new log.
I20260807 05:11:47.491110 22201 log.cc:824] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633: Log is configured to *not* fsync() on all Append() calls
I20260807 05:11:47.493489 22201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633: No bootstrap required, opened a new log
I20260807 05:11:47.490461 22208 master_runner.cc:186] Error getting master registration for 127.8.16.126:38149: OK, instance_id { permanent_uuid: "42476583a6654087a213a94098314633" instance_seqno: 1786079507013352 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 05:11:47.494185 22208 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 05:11:47.496718 22401 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3f1cd654dc974ed58c18bb041b6f162e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:11:47.496793 22401 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:11:47.497309 22201 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42476583a6654087a213a94098314633" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38149 } } peers { permanent_uuid: "3f1cd654dc974ed58c18bb041b6f162e" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } } peers { permanent_uuid: "97475af900cc4c8d8d7390851c6e297b" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } }
I20260807 05:11:47.497509 22201 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:11:47.497565 22201 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42476583a6654087a213a94098314633, State: Initialized, Role: FOLLOWER
I20260807 05:11:47.497972 22201 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42476583a6654087a213a94098314633" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38149 } } peers { permanent_uuid: "3f1cd654dc974ed58c18bb041b6f162e" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } } peers { permanent_uuid: "97475af900cc4c8d8d7390851c6e297b" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } }
I20260807 05:11:47.498386 22404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42476583a6654087a213a94098314633" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38149 } } peers { permanent_uuid: "3f1cd654dc974ed58c18bb041b6f162e" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } } peers { permanent_uuid: "97475af900cc4c8d8d7390851c6e297b" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } } }
I20260807 05:11:47.498543 22404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:11:47.499109 22201 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 05:11:47.504567 22415 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:11:47.504649 22415 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:11:47.508246 22122 master_runner.cc:263] No leader master found from master 42476583a6654087a213a94098314633
I20260807 05:11:47.508316 22122 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 05:11:47.509918 22143 tablet.cc:2369] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:11:47.628412 22404 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:11:47.628616 22404 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42476583a6654087a213a94098314633" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38149 } } peers { permanent_uuid: "3f1cd654dc974ed58c18bb041b6f162e" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } } peers { permanent_uuid: "97475af900cc4c8d8d7390851c6e297b" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } }
I20260807 05:11:47.629596 22404 leader_election.cc:288] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f1cd654dc974ed58c18bb041b6f162e (127.8.16.125:36121), 97475af900cc4c8d8d7390851c6e297b (127.8.16.124:42367)
I20260807 05:11:47.630290 22254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "42476583a6654087a213a94098314633" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1cd654dc974ed58c18bb041b6f162e" is_pre_election: true
I20260807 05:11:47.630597 22254 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3f1cd654dc974ed58c18bb041b6f162e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42476583a6654087a213a94098314633 in term 0.
I20260807 05:11:47.631152 22138 leader_election.cc:302] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f1cd654dc974ed58c18bb041b6f162e, 42476583a6654087a213a94098314633; no voters:
I20260807 05:11:47.631518 22404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:11:47.631613 22404 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:11:47.631659 22404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:11:47.632984 22404 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42476583a6654087a213a94098314633" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38149 } } peers { permanent_uuid: "3f1cd654dc974ed58c18bb041b6f162e" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } } peers { permanent_uuid: "97475af900cc4c8d8d7390851c6e297b" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } }
I20260807 05:11:47.633905 22254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "42476583a6654087a213a94098314633" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1cd654dc974ed58c18bb041b6f162e"
I20260807 05:11:47.634037 22254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f1cd654dc974ed58c18bb041b6f162e [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:11:47.635319 22254 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3f1cd654dc974ed58c18bb041b6f162e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42476583a6654087a213a94098314633 in term 1.
I20260807 05:11:47.635708 22138 leader_election.cc:302] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f1cd654dc974ed58c18bb041b6f162e, 42476583a6654087a213a94098314633; no voters:
I20260807 05:11:47.636603 22334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "42476583a6654087a213a94098314633" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97475af900cc4c8d8d7390851c6e297b" is_pre_election: true
I20260807 05:11:47.636932 22334 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42476583a6654087a213a94098314633 in term 0.
I20260807 05:11:47.637243 22333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "42476583a6654087a213a94098314633" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97475af900cc4c8d8d7390851c6e297b"
I20260807 05:11:47.637355 22333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:11:47.638659 22333 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42476583a6654087a213a94098314633 in term 1.
I20260807 05:11:47.639366 22404 leader_election.cc:288] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [CANDIDATE]: Term 1 election: Requested vote from peers 3f1cd654dc974ed58c18bb041b6f162e (127.8.16.125:36121), 97475af900cc4c8d8d7390851c6e297b (127.8.16.124:42367)
I20260807 05:11:47.639535 22404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:11:47.639649 22404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [term 1 LEADER]: Becoming Leader. State: Replica: 42476583a6654087a213a94098314633, State: Running, Role: LEADER
I20260807 05:11:47.639997 22404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42476583a6654087a213a94098314633" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38149 } } peers { permanent_uuid: "3f1cd654dc974ed58c18bb041b6f162e" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } } peers { permanent_uuid: "97475af900cc4c8d8d7390851c6e297b" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } }
I20260807 05:11:47.642218 22404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42476583a6654087a213a94098314633. Latest consensus state: current_term: 1 leader_uuid: "42476583a6654087a213a94098314633" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42476583a6654087a213a94098314633" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38149 } } peers { permanent_uuid: "3f1cd654dc974ed58c18bb041b6f162e" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } } peers { permanent_uuid: "97475af900cc4c8d8d7390851c6e297b" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } } }
I20260807 05:11:47.642334 22404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [sys.catalog]: This master's current role is: LEADER
I20260807 05:11:47.642724 22419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:11:47.643587 22419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:11:47.650614 22333 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b [term 1 FOLLOWER]: Refusing update from remote peer 42476583a6654087a213a94098314633: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:11:47.651458 22417 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97475af900cc4c8d8d7390851c6e297b" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 05:11:47.652026 22310 tablet.cc:2369] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:11:47.655695 22374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42476583a6654087a213a94098314633. Latest consensus state: current_term: 1 leader_uuid: "42476583a6654087a213a94098314633" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42476583a6654087a213a94098314633" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38149 } } peers { permanent_uuid: "3f1cd654dc974ed58c18bb041b6f162e" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } } peers { permanent_uuid: "97475af900cc4c8d8d7390851c6e297b" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } } }
I20260807 05:11:47.655833 22374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:11:47.656935 22254 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3f1cd654dc974ed58c18bb041b6f162e [term 1 FOLLOWER]: Refusing update from remote peer 42476583a6654087a213a94098314633: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:11:47.657572 22417 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1cd654dc974ed58c18bb041b6f162e" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:11:47.659689 22417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "42476583a6654087a213a94098314633" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42476583a6654087a213a94098314633" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38149 } } peers { permanent_uuid: "3f1cd654dc974ed58c18bb041b6f162e" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } } peers { permanent_uuid: "97475af900cc4c8d8d7390851c6e297b" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } } }
I20260807 05:11:47.659806 22417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [sys.catalog]: This master's current role is: LEADER
I20260807 05:11:47.659889 22420 mvcc.cc:205] Tried to move back new op lower bound from 7315781663328333824 to 7315781663296937984. Current Snapshot: MvccSnapshot[applied={T|T < 7315781663328333824}]
I20260807 05:11:47.661139 22421 mvcc.cc:205] Tried to move back new op lower bound from 7315781663328333824 to 7315781663296937984. Current Snapshot: MvccSnapshot[applied={T|T < 7315781663328333824}]
I20260807 05:11:47.661511 22374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "42476583a6654087a213a94098314633" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42476583a6654087a213a94098314633" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38149 } } peers { permanent_uuid: "3f1cd654dc974ed58c18bb041b6f162e" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } } peers { permanent_uuid: "97475af900cc4c8d8d7390851c6e297b" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } } }
I20260807 05:11:47.661633 22374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:11:47.661598 22389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f1cd654dc974ed58c18bb041b6f162e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42476583a6654087a213a94098314633. Latest consensus state: current_term: 1 leader_uuid: "42476583a6654087a213a94098314633" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42476583a6654087a213a94098314633" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38149 } } peers { permanent_uuid: "3f1cd654dc974ed58c18bb041b6f162e" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } } peers { permanent_uuid: "97475af900cc4c8d8d7390851c6e297b" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } } }
I20260807 05:11:47.661703 22389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f1cd654dc974ed58c18bb041b6f162e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:11:47.664752 22404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "42476583a6654087a213a94098314633" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42476583a6654087a213a94098314633" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38149 } } peers { permanent_uuid: "3f1cd654dc974ed58c18bb041b6f162e" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } } peers { permanent_uuid: "97475af900cc4c8d8d7390851c6e297b" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } } }
I20260807 05:11:47.664870 22404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633 [sys.catalog]: This master's current role is: LEADER
I20260807 05:11:47.665771 22419 catalog_manager.cc:1347] Generated new cluster ID: 436683704c114377b442b77026630c19
I20260807 05:11:47.665838 22419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:11:47.667390 22389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f1cd654dc974ed58c18bb041b6f162e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "42476583a6654087a213a94098314633" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42476583a6654087a213a94098314633" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 38149 } } peers { permanent_uuid: "3f1cd654dc974ed58c18bb041b6f162e" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 36121 } } peers { permanent_uuid: "97475af900cc4c8d8d7390851c6e297b" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 42367 } } }
I20260807 05:11:47.667496 22389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f1cd654dc974ed58c18bb041b6f162e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:11:47.668273 22424 mvcc.cc:205] Tried to move back new op lower bound from 7315781663328333824 to 7315781663296937984. Current Snapshot: MvccSnapshot[applied={T|T < 7315781663328333824}]
I20260807 05:11:47.675233 22419 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:11:47.676476 22419 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:11:47.686059 22419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42476583a6654087a213a94098314633: Generated new TSK 0
I20260807 05:11:47.686790 22419 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:11:48.469103 22386 catalog_manager.cc:1259] Loaded cluster ID: 436683704c114377b442b77026630c19
I20260807 05:11:48.469195 22386 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b: loading cluster ID for follower catalog manager: success
I20260807 05:11:48.470857 22386 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b: acquiring CA information for follower catalog manager: success
I20260807 05:11:48.472002 22386 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 97475af900cc4c8d8d7390851c6e297b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 05:11:48.498157 22401 catalog_manager.cc:1259] Loaded cluster ID: 436683704c114377b442b77026630c19
I20260807 05:11:48.498246 22401 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3f1cd654dc974ed58c18bb041b6f162e: loading cluster ID for follower catalog manager: success
I20260807 05:11:48.499969 22401 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3f1cd654dc974ed58c18bb041b6f162e: acquiring CA information for follower catalog manager: success
I20260807 05:11:48.501180 22401 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3f1cd654dc974ed58c18bb041b6f162e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 05:11:48.751217 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 22122
I20260807 05:11:48.758352 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 22208
I20260807 05:11:48.764611 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 22288
2026-08-07T05:11:48Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3026 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:11:48 dist-test-slave-49h3 krb5kdc[22435](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:11:48 dist-test-slave-49h3 krb5kdc[22435](info): set up 2 sockets
Aug 07 05:11:48 dist-test-slave-49h3 krb5kdc[22435](info): commencing operation
krb5kdc: starting...
W20260807 05:11:49.745275 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.959s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:11:49 dist-test-slave-49h3 krb5kdc[22435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:11:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:11:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:11:49.857637 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:35169
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:37435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:35169
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:11:49.967783 22449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:49.968053 22449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:49.968122 22449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:49.972391 22449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:11:49.972481 22449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:49.972512 22449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:49.972532 22449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:11:49.972553 22449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:11:49.978243 22449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:37435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:35169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:35169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:49.979401 22449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:49.980557 22449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:11:49.988731 22455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:49.988736 22459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:49.988830 22449 server_base.cc:1034] running on GCE node
W20260807 05:11:49.988731 22454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:49.989550 22449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:49.990572 22449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:49.991747 22449 hybrid_clock.cc:648] HybridClock initialized: now 1786079509991732 us; error 28 us; skew 500 ppm
Aug 07 05:11:49 dist-test-slave-49h3 krb5kdc[22435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079509, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:49.994443 22449 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:11:49.995261 22449 webserver.cc:460] Webserver started at http://127.8.16.126:32817/ using document root <none> and password file <none>
I20260807 05:11:49.995772 22449 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:49.995836 22449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:11:49.996014 22449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:11:49.998042 22449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "36c76d8648b146e6a271809ed7656e06"
format_stamp: "Formatted at 2026-08-07 05:11:49 on dist-test-slave-49h3"
server_key: "7c848d97cf1c39de4b82d855c128a9ec"
server_key_iv: "50714f12f317a1fe39f4d5847865bc05"
server_key_version: "encryptionkey@0"
I20260807 05:11:49.998555 22449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "36c76d8648b146e6a271809ed7656e06"
format_stamp: "Formatted at 2026-08-07 05:11:49 on dist-test-slave-49h3"
server_key: "7c848d97cf1c39de4b82d855c128a9ec"
server_key_iv: "50714f12f317a1fe39f4d5847865bc05"
server_key_version: "encryptionkey@0"
I20260807 05:11:50.002045 22449 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:11:50.004206 22466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:50.005376 22449 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:11:50.005502 22449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "36c76d8648b146e6a271809ed7656e06"
format_stamp: "Formatted at 2026-08-07 05:11:49 on dist-test-slave-49h3"
server_key: "7c848d97cf1c39de4b82d855c128a9ec"
server_key_iv: "50714f12f317a1fe39f4d5847865bc05"
server_key_version: "encryptionkey@0"
I20260807 05:11:50.005594 22449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:50.020211 22449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:50.023398 22449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:50.023587 22449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:50.030200 22449 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:35169
I20260807 05:11:50.030244 22519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:35169 every 8 connection(s)
I20260807 05:11:50.031123 22449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:11:50.033178 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 22449
I20260807 05:11:50.033291 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:11:50.033605 8257 external_mini_cluster.cc:1428] Setting key 56aea7bde53613f461a8f27feb0283c6
I20260807 05:11:50.033874 22520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:11:50.040836 22520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36c76d8648b146e6a271809ed7656e06: Bootstrap starting.
Aug 07 05:11:50 dist-test-slave-49h3 krb5kdc[22435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:11:50.043094 22520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36c76d8648b146e6a271809ed7656e06: Neither blocks nor log segments found. Creating new log.
I20260807 05:11:50.043964 22520 log.cc:824] T 00000000000000000000000000000000 P 36c76d8648b146e6a271809ed7656e06: Log is configured to *not* fsync() on all Append() calls
I20260807 05:11:50.046792 22520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36c76d8648b146e6a271809ed7656e06: No bootstrap required, opened a new log
I20260807 05:11:50.050041 22523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:50.034918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35530 (local address 127.8.16.126:35169)
0807 05:11:50.035339 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:50.035357 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:50.035400 (+ 43us) server_negotiation.cc:407] Connection header received
0807 05:11:50.036134 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:50.036157 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:50.036561 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:50.036925 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:50.037610 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:50.039055 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:50.039864 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:50.040528 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:50.040694 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:50.043122 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:50.043129 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:50.043145 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:50.043178 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:50.045190 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:50.045738 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:50.045744 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:50.045750 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:50.045911 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:50.046365 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:50.046370 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:50.046372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:50.047296 (+ 924us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:50.048456 (+ 1160us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:50.048624 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 05:11:50.048956 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":122,"threads_started":1}
I20260807 05:11:50.050994 22520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36c76d8648b146e6a271809ed7656e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c76d8648b146e6a271809ed7656e06" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35169 } }
I20260807 05:11:50.051266 22520 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36c76d8648b146e6a271809ed7656e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:11:50.051316 22520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36c76d8648b146e6a271809ed7656e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36c76d8648b146e6a271809ed7656e06, State: Initialized, Role: FOLLOWER
I20260807 05:11:50.051753 22520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36c76d8648b146e6a271809ed7656e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c76d8648b146e6a271809ed7656e06" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35169 } }
I20260807 05:11:50.051892 22520 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36c76d8648b146e6a271809ed7656e06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:11:50.051963 22520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36c76d8648b146e6a271809ed7656e06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:11:50.052058 22520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36c76d8648b146e6a271809ed7656e06 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:11:50.053157 22520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36c76d8648b146e6a271809ed7656e06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c76d8648b146e6a271809ed7656e06" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35169 } }
I20260807 05:11:50.053527 22520 leader_election.cc:302] T 00000000000000000000000000000000 P 36c76d8648b146e6a271809ed7656e06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36c76d8648b146e6a271809ed7656e06; no voters:
I20260807 05:11:50.053833 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:11:50.053859 22520 leader_election.cc:288] T 00000000000000000000000000000000 P 36c76d8648b146e6a271809ed7656e06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:11:50.053970 22525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36c76d8648b146e6a271809ed7656e06 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:11:50.054191 22525 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36c76d8648b146e6a271809ed7656e06 [term 1 LEADER]: Becoming Leader. State: Replica: 36c76d8648b146e6a271809ed7656e06, State: Running, Role: LEADER
I20260807 05:11:50.054495 22525 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36c76d8648b146e6a271809ed7656e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c76d8648b146e6a271809ed7656e06" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35169 } }
I20260807 05:11:50.055163 22520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36c76d8648b146e6a271809ed7656e06 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:11:50.055656 22525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36c76d8648b146e6a271809ed7656e06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36c76d8648b146e6a271809ed7656e06. Latest consensus state: current_term: 1 leader_uuid: "36c76d8648b146e6a271809ed7656e06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c76d8648b146e6a271809ed7656e06" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35169 } } }
I20260807 05:11:50.055771 22525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36c76d8648b146e6a271809ed7656e06 [sys.catalog]: This master's current role is: LEADER
I20260807 05:11:50.056162 22527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36c76d8648b146e6a271809ed7656e06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36c76d8648b146e6a271809ed7656e06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c76d8648b146e6a271809ed7656e06" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 35169 } } }
I20260807 05:11:50.056324 22527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36c76d8648b146e6a271809ed7656e06 [sys.catalog]: This master's current role is: LEADER
I20260807 05:11:50.056556 22530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:11:50.059612 22530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:11:50.065383 22530 catalog_manager.cc:1347] Generated new cluster ID: 33ebd15a4dfe4785a5ce863e168ef887
I20260807 05:11:50.065480 22530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:11:50.089439 22530 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:11:50.090822 22530 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:11:50.097087 22530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36c76d8648b146e6a271809ed7656e06: Generated new TSK 0
I20260807 05:11:50.097839 22530 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:11:50.110877 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 22449
2026-08-07T05:11:50Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1345 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:11:50 dist-test-slave-49h3 krb5kdc[22548](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:11:50 dist-test-slave-49h3 krb5kdc[22548](info): set up 2 sockets
Aug 07 05:11:50 dist-test-slave-49h3 krb5kdc[22548](info): commencing operation
krb5kdc: starting...
W20260807 05:11:51.093176 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.962s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:11:51 dist-test-slave-49h3 krb5kdc[22548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:11:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:11:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:11:51.209398 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:44131
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:35985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:44131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:11:51.318539 22561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:51.318821 22561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:51.318874 22561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:51.322919 22561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:11:51.322992 22561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:51.323017 22561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:51.323038 22561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:11:51.323074 22561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:11:51.328409 22561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:35985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:44131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:44131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:51.329519 22561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:51.330567 22561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:11:51.338704 22570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:51.338752 22566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:51.338716 22567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:51.338917 22561 server_base.cc:1034] running on GCE node
I20260807 05:11:51.339643 22561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:51.340677 22561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:51.341817 22561 hybrid_clock.cc:648] HybridClock initialized: now 1786079511341792 us; error 40 us; skew 500 ppm
Aug 07 05:11:51 dist-test-slave-49h3 krb5kdc[22548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079511, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:51.349368 22561 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:11:51.350301 22561 webserver.cc:460] Webserver started at http://127.8.16.126:34171/ using document root <none> and password file <none>
I20260807 05:11:51.350850 22561 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:51.350922 22561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:11:51.351132 22561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:11:51.353178 22561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "6c5c1de2736e4027a7a6b15d06e19a1e"
format_stamp: "Formatted at 2026-08-07 05:11:51 on dist-test-slave-49h3"
server_key: "442cc7ac7135d34c381312558e5c6841"
server_key_iv: "7d0873c9f1275e5b520cce454bc753b2"
server_key_version: "encryptionkey@0"
I20260807 05:11:51.353747 22561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "6c5c1de2736e4027a7a6b15d06e19a1e"
format_stamp: "Formatted at 2026-08-07 05:11:51 on dist-test-slave-49h3"
server_key: "442cc7ac7135d34c381312558e5c6841"
server_key_iv: "7d0873c9f1275e5b520cce454bc753b2"
server_key_version: "encryptionkey@0"
I20260807 05:11:51.357343 22561 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 05:11:51.359690 22577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:51.360860 22561 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:11:51.360977 22561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "6c5c1de2736e4027a7a6b15d06e19a1e"
format_stamp: "Formatted at 2026-08-07 05:11:51 on dist-test-slave-49h3"
server_key: "442cc7ac7135d34c381312558e5c6841"
server_key_iv: "7d0873c9f1275e5b520cce454bc753b2"
server_key_version: "encryptionkey@0"
I20260807 05:11:51.361079 22561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:51.399021 22561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:51.402668 22561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:51.402868 22561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:51.409781 22561 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:44131
I20260807 05:11:51.410030 22633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:44131 every 8 connection(s)
I20260807 05:11:51.410740 22561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:11:51.413362 22634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:11:51.415230 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 22561
I20260807 05:11:51.415349 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:11:51.415613 8257 external_mini_cluster.cc:1428] Setting key 6e06ed865b1ff9661239387fa476426b
I20260807 05:11:51.419796 22634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e: Bootstrap starting.
I20260807 05:11:51.422330 22634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e: Neither blocks nor log segments found. Creating new log.
I20260807 05:11:51.423183 22634 log.cc:824] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e: Log is configured to *not* fsync() on all Append() calls
Aug 07 05:11:51 dist-test-slave-49h3 krb5kdc[22548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:11:51.425216 22634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e: No bootstrap required, opened a new log
I20260807 05:11:51.428992 22634 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44131 } }
I20260807 05:11:51.429210 22637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:51.416918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45754 (local address 127.8.16.126:44131)
0807 05:11:51.417322 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:51.417337 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:51.417373 (+ 36us) server_negotiation.cc:407] Connection header received
0807 05:11:51.418038 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:51.418054 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:51.418375 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:51.418637 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:51.419552 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:51.420523 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:51.421704 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:51.422223 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:51.422344 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:51.424829 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:51.424836 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:51.424851 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:51.424881 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:51.426553 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:51.427039 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:51.427043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:51.427048 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:51.427154 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:51.427403 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:51.427405 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:51.427406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:51.427779 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:51.427931 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:51.428192 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 05:11:51.428475 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":106,"threads_started":1}
I20260807 05:11:51.429219 22634 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:11:51.429333 22634 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c5c1de2736e4027a7a6b15d06e19a1e, State: Initialized, Role: FOLLOWER
I20260807 05:11:51.429777 22634 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44131 } }
I20260807 05:11:51.429908 22634 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:11:51.429972 22634 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:11:51.430068 22634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:11:51.431169 22634 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44131 } }
I20260807 05:11:51.431492 22634 leader_election.cc:302] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c5c1de2736e4027a7a6b15d06e19a1e; no voters:
I20260807 05:11:51.431790 22634 leader_election.cc:288] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:11:51.431872 22639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:11:51.432081 22639 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 1 LEADER]: Becoming Leader. State: Replica: 6c5c1de2736e4027a7a6b15d06e19a1e, State: Running, Role: LEADER
I20260807 05:11:51.432386 22639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44131 } }
I20260807 05:11:51.432883 22634 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:11:51.434032 22641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c5c1de2736e4027a7a6b15d06e19a1e. Latest consensus state: current_term: 1 leader_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44131 } } }
I20260807 05:11:51.434146 22641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [sys.catalog]: This master's current role is: LEADER
I20260807 05:11:51.434389 22640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44131 } } }
I20260807 05:11:51.434475 22640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [sys.catalog]: This master's current role is: LEADER
I20260807 05:11:51.434520 22648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:11:51.437006 22648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:11:51.441862 22648 catalog_manager.cc:1347] Generated new cluster ID: e9b118e990054a0aa4e104debe822d61
I20260807 05:11:51.441968 22648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:11:51.450237 22648 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:11:51.451169 22648 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:11:51.465638 22648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e: Generated new TSK 0
I20260807 05:11:51.466338 22648 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
I20260807 05:11:51.496217 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:44131
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:35985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:11:51.606103 22660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:51.606390 22660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:51.606436 22660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:51.610450 22660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:51.610522 22660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:51.610594 22660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:11:51.615899 22660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:35985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:44131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:51.616963 22660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:51.618014 22660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:11:51.627167 22665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:51.627127 22670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:51.627280 22660 server_base.cc:1034] running on GCE node
W20260807 05:11:51.627127 22666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:51.627818 22660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:51.628372 22660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:51.629516 22660 hybrid_clock.cc:648] HybridClock initialized: now 1786079511629498 us; error 27 us; skew 500 ppm
Aug 07 05:11:51 dist-test-slave-49h3 krb5kdc[22548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079511, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:51.632090 22660 init.cc:377] Logged in from keytab as kudu/127.8.16.65@KRBTEST.COM (short username kudu)
I20260807 05:11:51.632906 22660 webserver.cc:460] Webserver started at http://127.8.16.65:42171/ using document root <none> and password file <none>
I20260807 05:11:51.633492 22660 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:51.633562 22660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:11:51.633735 22660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:11:51.635735 22660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "88d6b5da8aba4197a622da1cbc66023d"
format_stamp: "Formatted at 2026-08-07 05:11:51 on dist-test-slave-49h3"
server_key: "12c85cb5b76908b1f9af9f48d53c4897"
server_key_iv: "bf9fc055a5ab96ec809a178d9416b7b2"
server_key_version: "encryptionkey@0"
I20260807 05:11:51.636221 22660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "88d6b5da8aba4197a622da1cbc66023d"
format_stamp: "Formatted at 2026-08-07 05:11:51 on dist-test-slave-49h3"
server_key: "12c85cb5b76908b1f9af9f48d53c4897"
server_key_iv: "bf9fc055a5ab96ec809a178d9416b7b2"
server_key_version: "encryptionkey@0"
I20260807 05:11:51.639637 22660 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:11:51.641824 22677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:51.642942 22660 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:11:51.643038 22660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "88d6b5da8aba4197a622da1cbc66023d"
format_stamp: "Formatted at 2026-08-07 05:11:51 on dist-test-slave-49h3"
server_key: "12c85cb5b76908b1f9af9f48d53c4897"
server_key_iv: "bf9fc055a5ab96ec809a178d9416b7b2"
server_key_version: "encryptionkey@0"
I20260807 05:11:51.643133 22660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:51.667965 22660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:51.671139 22660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:51.671329 22660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:51.671907 22660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:11:51.672716 22660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:11:51.672766 22660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:51.672807 22660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:11:51.672837 22660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:51.680842 22660 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:32949
I20260807 05:11:51.680894 22801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:32949 every 8 connection(s)
I20260807 05:11:51.681775 22660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:11:51.682680 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 22660
I20260807 05:11:51.682818 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:11:51.683136 8257 external_mini_cluster.cc:1428] Setting key 38e2769f9d43229bd385b562ff1662bd
Aug 07 05:11:51 dist-test-slave-49h3 krb5kdc[22548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079511, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.66@KRBTEST.COM" created.
I20260807 05:11:51.695932 22637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:51.683659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:42215 (local address 127.8.16.126:44131)
0807 05:11:51.683824 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:51.683828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:51.684964 (+ 1136us) server_negotiation.cc:407] Connection header received
0807 05:11:51.686019 (+ 1055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:51.686022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:51.686080 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:51.686147 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:51.687426 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:51.688024 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:51.689662 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:51.689894 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:51.689972 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:51.692611 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:51.692614 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:51.692616 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:51.692640 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:51.693978 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:51.694446 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:51.694449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:51.694450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:51.694521 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:51.694837 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:51.694840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:51.694842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:51.695017 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:51.695101 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:51.695623 (+ 522us) server_negotiation.cc:299] Negotiation successful
0807 05:11:51.695764 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 05:11:51.696730 22806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:51.684080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.65:42215)
0807 05:11:51.684802 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:51.684859 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:51.685823 (+ 964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:51.686325 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:51.686333 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:51.686797 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:51.687272 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:51.687286 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:51.688170 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:51.688173 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:51.689557 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:51.689569 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:51.690049 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:51.690053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:51.690261 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:51.690843 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:51.690864 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:51.692457 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:51.694085 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:51.694091 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:51.694107 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:51.694355 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:51.694602 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:51.694605 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:51.694608 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:51.694749 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:51.695115 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:51.695121 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:51.695360 (+ 239us) client_negotiation.cc:769] Sending connection context
0807 05:11:51.695562 (+ 202us) client_negotiation.cc:240] Negotiation successful
0807 05:11:51.695927 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":131,"threads_started":1}
I20260807 05:11:51.697978 22803 heartbeater.cc:347] Connected to a master server at 127.8.16.126:44131
I20260807 05:11:51.698249 22803 heartbeater.cc:464] Registering TS with master...
I20260807 05:11:51.698788 22803 heartbeater.cc:511] Master 127.8.16.126:44131 requested a full tablet report, sending...
I20260807 05:11:51.700256 22596 ts_manager.cc:194] Registered new tserver with Master: 88d6b5da8aba4197a622da1cbc66023d (127.8.16.65:32949)
I20260807 05:11:51.701355 22596 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:42215
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
I20260807 05:11:51.710160 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:44131
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:35985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:11:51.818281 22809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:51.818560 22809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:51.818605 22809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:51.822649 22809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:51.822722 22809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:51.822799 22809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:11:51.828153 22809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:35985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:44131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:51.829232 22809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:51.830269 22809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:11:51.839610 22814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:51.839605 22815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:51.839605 22817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:51.839859 22809 server_base.cc:1034] running on GCE node
I20260807 05:11:51.840229 22809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:51.840832 22809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:51.841987 22809 hybrid_clock.cc:648] HybridClock initialized: now 1786079511841974 us; error 25 us; skew 500 ppm
Aug 07 05:11:51 dist-test-slave-49h3 krb5kdc[22548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079511, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:51.844484 22809 init.cc:377] Logged in from keytab as kudu/127.8.16.66@KRBTEST.COM (short username kudu)
I20260807 05:11:51.845326 22809 webserver.cc:460] Webserver started at http://127.8.16.66:46183/ using document root <none> and password file <none>
I20260807 05:11:51.845872 22809 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:51.845952 22809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:11:51.846184 22809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:11:51.848161 22809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "93ccbf8081324583b623143d78125e41"
format_stamp: "Formatted at 2026-08-07 05:11:51 on dist-test-slave-49h3"
server_key: "12b862660c003cdbc8804c2742ca4025"
server_key_iv: "99a040659bf31996dcf05def6e548ecb"
server_key_version: "encryptionkey@0"
I20260807 05:11:51.848670 22809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "93ccbf8081324583b623143d78125e41"
format_stamp: "Formatted at 2026-08-07 05:11:51 on dist-test-slave-49h3"
server_key: "12b862660c003cdbc8804c2742ca4025"
server_key_iv: "99a040659bf31996dcf05def6e548ecb"
server_key_version: "encryptionkey@0"
I20260807 05:11:51.852195 22809 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:11:51.854364 22824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:51.855422 22809 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:11:51.855521 22809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "93ccbf8081324583b623143d78125e41"
format_stamp: "Formatted at 2026-08-07 05:11:51 on dist-test-slave-49h3"
server_key: "12b862660c003cdbc8804c2742ca4025"
server_key_iv: "99a040659bf31996dcf05def6e548ecb"
server_key_version: "encryptionkey@0"
I20260807 05:11:51.855607 22809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:51.878239 22809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:51.880934 22809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:51.881126 22809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:51.881767 22809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:11:51.882587 22809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:11:51.882635 22809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:51.882688 22809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:11:51.882718 22809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:51.891042 22809 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:46281
I20260807 05:11:51.891070 22949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:46281 every 8 connection(s)
I20260807 05:11:51.891906 22809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:11:51.896198 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 22809
I20260807 05:11:51.896328 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:11:51.896641 8257 external_mini_cluster.cc:1428] Setting key 3892484c262a16f1e2aa660d68e06a0f
Aug 07 05:11:51 dist-test-slave-49h3 krb5kdc[22548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079511, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:11:51.905514 22637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:51.893704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:59889 (local address 127.8.16.126:44131)
0807 05:11:51.893865 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:51.893869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:51.894891 (+ 1022us) server_negotiation.cc:407] Connection header received
0807 05:11:51.895671 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:51.895673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:51.895719 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:51.895792 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:51.897235 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:51.897925 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:51.899403 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:51.899624 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:51.899701 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:51.902250 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:51.902254 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:51.902256 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:51.902280 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:51.903569 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:51.904033 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:51.904036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:51.904038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:51.904110 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:51.904411 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:51.904414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:51.904416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:51.904584 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:51.904665 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:51.905203 (+ 538us) server_negotiation.cc:299] Negotiation successful
0807 05:11:51.905319 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
WARNING: no policy specified for kudu/127.8.16.67@KRBTEST.COM; defaulting to no policy
I20260807 05:11:51.906350 22952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:51.894111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.66:59889)
0807 05:11:51.894743 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:51.894782 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:51.895510 (+ 728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:51.895940 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:51.895949 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:51.896423 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:51.897086 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:51.897102 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:51.898044 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:51.898047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:51.899289 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:51.899300 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:51.899760 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:51.899763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:51.899945 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:51.900589 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:51.900606 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:51.902094 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:51.903678 (+ 1584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:51.903684 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:51.903699 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:51.903942 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:51.904188 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:51.904192 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:51.904194 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:51.904326 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:51.904686 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:51.904693 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:51.904953 (+ 260us) client_negotiation.cc:769] Sending connection context
0807 05:11:51.905144 (+ 191us) client_negotiation.cc:240] Negotiation successful
0807 05:11:51.905487 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":111,"threads_started":1}
I20260807 05:11:51.907475 22950 heartbeater.cc:347] Connected to a master server at 127.8.16.126:44131
I20260807 05:11:51.907712 22950 heartbeater.cc:464] Registering TS with master...
I20260807 05:11:51.908254 22950 heartbeater.cc:511] Master 127.8.16.126:44131 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.67@KRBTEST.COM" created.
I20260807 05:11:51.909353 22596 ts_manager.cc:194] Registered new tserver with Master: 93ccbf8081324583b623143d78125e41 (127.8.16.66:46281)
I20260807 05:11:51.910069 22596 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:59889
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
I20260807 05:11:51.921633 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:44131
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:35985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:11:52.032477 22955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:52.032755 22955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:52.032797 22955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:52.036993 22955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:52.037057 22955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:52.037133 22955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:11:52.042376 22955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:35985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:44131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:52.043413 22955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:52.044413 22955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:11:52.053537 22960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:52.053593 22961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:52.053545 22963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:52.053951 22955 server_base.cc:1034] running on GCE node
I20260807 05:11:52.054330 22955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:52.054929 22955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:52.056082 22955 hybrid_clock.cc:648] HybridClock initialized: now 1786079512056061 us; error 34 us; skew 500 ppm
Aug 07 05:11:52 dist-test-slave-49h3 krb5kdc[22548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079512, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:52.058789 22955 init.cc:377] Logged in from keytab as kudu/127.8.16.67@KRBTEST.COM (short username kudu)
I20260807 05:11:52.059712 22955 webserver.cc:460] Webserver started at http://127.8.16.67:39691/ using document root <none> and password file <none>
I20260807 05:11:52.060233 22955 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:52.060299 22955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:11:52.060489 22955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:11:52.062546 22955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "e9e3de1593ae48d9b4223fc56c69d5aa"
format_stamp: "Formatted at 2026-08-07 05:11:52 on dist-test-slave-49h3"
server_key: "42dd5bb737a9c6f9196ce5b08ebfd628"
server_key_iv: "8c8d414f2c91c77e94a024034b3312cb"
server_key_version: "encryptionkey@0"
I20260807 05:11:52.063100 22955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "e9e3de1593ae48d9b4223fc56c69d5aa"
format_stamp: "Formatted at 2026-08-07 05:11:52 on dist-test-slave-49h3"
server_key: "42dd5bb737a9c6f9196ce5b08ebfd628"
server_key_iv: "8c8d414f2c91c77e94a024034b3312cb"
server_key_version: "encryptionkey@0"
I20260807 05:11:52.066887 22955 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 05:11:52.069229 22970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:52.070695 22955 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:11:52.070806 22955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "e9e3de1593ae48d9b4223fc56c69d5aa"
format_stamp: "Formatted at 2026-08-07 05:11:52 on dist-test-slave-49h3"
server_key: "42dd5bb737a9c6f9196ce5b08ebfd628"
server_key_iv: "8c8d414f2c91c77e94a024034b3312cb"
server_key_version: "encryptionkey@0"
I20260807 05:11:52.070906 22955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:52.095779 22955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:52.098971 22955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:52.099169 22955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:52.099761 22955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:11:52.100572 22955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:11:52.100639 22955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:52.100694 22955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:11:52.100720 22955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:52.109306 22955 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:46527
I20260807 05:11:52.109344 23091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:46527 every 8 connection(s)
I20260807 05:11:52.110210 22955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:11:52.118199 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 22955
I20260807 05:11:52.118320 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:11:52.118562 8257 external_mini_cluster.cc:1428] Setting key 68f7719d1d83ecd33346cf9aa495fc02
Aug 07 05:11:52 dist-test-slave-49h3 krb5kdc[22548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079512, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:11:52.123101 22637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:52.111886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:48101 (local address 127.8.16.126:44131)
0807 05:11:52.112011 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:52.112015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:52.112995 (+ 980us) server_negotiation.cc:407] Connection header received
0807 05:11:52.113901 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:52.113904 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:52.113965 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:52.114042 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:52.115193 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.115725 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.116840 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.117057 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.117143 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:52.119718 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:52.119721 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:52.119722 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:52.119748 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:52.121078 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:52.121600 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:52.121603 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:52.121604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:52.121680 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:52.122002 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:52.122004 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:52.122005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:52.122194 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:52.122278 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:52.122785 (+ 507us) server_negotiation.cc:299] Negotiation successful
0807 05:11:52.122939 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 05:11:52.123869 23094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:52.112223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.67:48101)
0807 05:11:52.112844 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:52.112885 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:52.113696 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:52.114223 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:52.114229 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:52.114582 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:52.115045 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:52.115058 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.115844 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.115847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:52.116723 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:52.116732 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.117206 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.117210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:52.117368 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:52.117981 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:52.117998 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:52.119555 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:52.121191 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:52.121197 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:52.121214 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:52.121503 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:52.121761 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:52.121765 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:52.121767 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:52.121906 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:52.122290 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:52.122297 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:52.122547 (+ 250us) client_negotiation.cc:769] Sending connection context
0807 05:11:52.122711 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 05:11:52.123047 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":110,"threads_started":1}
I20260807 05:11:52.124892 23092 heartbeater.cc:347] Connected to a master server at 127.8.16.126:44131
I20260807 05:11:52.125125 23092 heartbeater.cc:464] Registering TS with master...
I20260807 05:11:52.125622 23092 heartbeater.cc:511] Master 127.8.16.126:44131 requested a full tablet report, sending...
I20260807 05:11:52.126590 22596 ts_manager.cc:194] Registered new tserver with Master: e9e3de1593ae48d9b4223fc56c69d5aa (127.8.16.67:46527)
I20260807 05:11:52.127151 22596 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:48101
I20260807 05:11:52.132352 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:11:52.140961 22637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:52.134416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45760 (local address 127.8.16.126:44131)
0807 05:11:52.134532 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:52.134535 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:52.134684 (+ 149us) server_negotiation.cc:407] Connection header received
0807 05:11:52.134810 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:52.134812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:52.134852 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:52.134909 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:52.135523 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.136027 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.136768 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.136975 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.137047 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:52.138280 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:52.138284 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:52.138285 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:52.138308 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:52.139427 (+ 1119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:52.139822 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:52.139825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:52.139827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:52.139922 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:52.140115 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:52.140117 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:52.140118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:52.140329 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:52.140429 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:52.140743 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 05:11:52.140842 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 05:11:52.143633 22596 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45760:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:11:52.146680 22596 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:11:52.160553 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:52.154589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60118 (local address 127.8.16.67:46527)
0807 05:11:52.155959 (+ 1370us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:52.155967 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:52.155984 (+ 17us) server_negotiation.cc:407] Connection header received
0807 05:11:52.156040 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:52.156044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:52.156202 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:52.156575 (+ 373us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:11:52.156697 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.157379 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.159037 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.159859 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.160026 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:52.160169 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:52.160304 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 05:11:52.160401 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1287,"thread_start_us":147,"threads_started":1}
I20260807 05:11:52.160553 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:52.154434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:46527 (local address 127.0.0.1:60118)
0807 05:11:52.155094 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:52.155350 (+ 256us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:52.155434 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:52.156343 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:52.156345 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:52.156369 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:11:52.156462 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:52.156468 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.157526 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.157529 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:52.158927 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:52.158936 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.159984 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.159987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:52.160103 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:52.160148 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 05:11:52.160256 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 05:11:52.160388 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":39,"spinlock_wait_cycles":768,"thread_start_us":89,"threads_started":1}
I20260807 05:11:52.160568 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:52.155160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:32949 (local address 127.0.0.1:59570)
0807 05:11:52.155664 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:52.155677 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:52.155753 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:52.156304 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:52.156307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:52.156324 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:11:52.156435 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:52.156442 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.158057 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.158060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:52.159063 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:52.159074 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.160041 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.160044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:52.160138 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:52.160171 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 05:11:52.160229 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 05:11:52.160381 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":3,"spinlock_wait_cycles":896,"thread_start_us":89,"threads_started":1}
I20260807 05:11:52.160569 23104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:52.154545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:46281 (local address 127.0.0.1:48746)
0807 05:11:52.155281 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:52.155299 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:52.155425 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:52.156194 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:52.156199 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:52.156221 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:11:52.156362 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:52.156369 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.157936 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.157940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:52.159187 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:52.159195 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.160244 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.160247 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:52.160324 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:52.160343 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 05:11:52.160404 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 05:11:52.160457 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"mutex_wait_us":57,"thread_start_us":104,"threads_started":1}
I20260807 05:11:52.160877 23106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:52.154897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48746 (local address 127.8.16.66:46281)
0807 05:11:52.155699 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:52.155711 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:52.155744 (+ 33us) server_negotiation.cc:407] Connection header received
0807 05:11:52.155820 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:52.155825 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:52.156026 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:52.156180 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:11:52.156476 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.157686 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.159284 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.159867 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.159961 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:52.160107 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:52.160633 (+ 526us) server_negotiation.cc:299] Negotiation successful
0807 05:11:52.160743 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":134,"threads_started":1}
I20260807 05:11:52.161542 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:52.155563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59570 (local address 127.8.16.65:32949)
0807 05:11:52.155900 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:52.155914 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:52.155933 (+ 19us) server_negotiation.cc:407] Connection header received
0807 05:11:52.155994 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:52.155998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:52.156149 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:52.156582 (+ 433us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:11:52.156721 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.157831 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.159186 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.159919 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.161119 (+ 1200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:52.161263 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:52.161307 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 05:11:52.161388 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":85,"threads_started":1}
I20260807 05:11:52.163523 22877 tablet_service.cc:1467] Processing CreateTablet for tablet e77bfe8760214d54bd10d9af7d025190 (DEFAULT_TABLE table=test-table [id=60d7987ad8704365b9e2f2662e0f2290]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:11:52.163663 23021 tablet_service.cc:1467] Processing CreateTablet for tablet e77bfe8760214d54bd10d9af7d025190 (DEFAULT_TABLE table=test-table [id=60d7987ad8704365b9e2f2662e0f2290]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:11:52.163904 22732 tablet_service.cc:1467] Processing CreateTablet for tablet e77bfe8760214d54bd10d9af7d025190 (DEFAULT_TABLE table=test-table [id=60d7987ad8704365b9e2f2662e0f2290]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:11:52.164537 22877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e77bfe8760214d54bd10d9af7d025190. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:11:52.164739 23021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e77bfe8760214d54bd10d9af7d025190. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:11:52.164722 22732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e77bfe8760214d54bd10d9af7d025190. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:11:52.170825 23109 tablet_bootstrap.cc:492] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41: Bootstrap starting.
I20260807 05:11:52.171473 23110 tablet_bootstrap.cc:492] T e77bfe8760214d54bd10d9af7d025190 P e9e3de1593ae48d9b4223fc56c69d5aa: Bootstrap starting.
I20260807 05:11:52.172155 23111 tablet_bootstrap.cc:492] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d: Bootstrap starting.
I20260807 05:11:52.173545 23109 tablet_bootstrap.cc:654] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41: Neither blocks nor log segments found. Creating new log.
I20260807 05:11:52.173817 23110 tablet_bootstrap.cc:654] T e77bfe8760214d54bd10d9af7d025190 P e9e3de1593ae48d9b4223fc56c69d5aa: Neither blocks nor log segments found. Creating new log.
I20260807 05:11:52.173882 23111 tablet_bootstrap.cc:654] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d: Neither blocks nor log segments found. Creating new log.
I20260807 05:11:52.174474 23109 log.cc:824] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41: Log is configured to *not* fsync() on all Append() calls
I20260807 05:11:52.174499 23110 log.cc:824] T e77bfe8760214d54bd10d9af7d025190 P e9e3de1593ae48d9b4223fc56c69d5aa: Log is configured to *not* fsync() on all Append() calls
I20260807 05:11:52.175026 23111 log.cc:824] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d: Log is configured to *not* fsync() on all Append() calls
I20260807 05:11:52.176321 23109 tablet_bootstrap.cc:492] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41: No bootstrap required, opened a new log
I20260807 05:11:52.176425 23110 tablet_bootstrap.cc:492] T e77bfe8760214d54bd10d9af7d025190 P e9e3de1593ae48d9b4223fc56c69d5aa: No bootstrap required, opened a new log
I20260807 05:11:52.176496 23109 ts_tablet_manager.cc:1397] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 05:11:52.176599 23110 ts_tablet_manager.cc:1397] T e77bfe8760214d54bd10d9af7d025190 P e9e3de1593ae48d9b4223fc56c69d5aa: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:11:52.176854 23111 tablet_bootstrap.cc:492] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d: No bootstrap required, opened a new log
I20260807 05:11:52.177013 23111 ts_tablet_manager.cc:1397] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 05:11:52.179343 23109 raft_consensus.cc:348] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:52.179558 23109 raft_consensus.cc:374] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:11:52.179627 23109 raft_consensus.cc:729] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93ccbf8081324583b623143d78125e41, State: Initialized, Role: FOLLOWER
I20260807 05:11:52.180022 23109 consensus_queue.cc:260] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:52.180586 22950 heartbeater.cc:503] Master 127.8.16.126:44131 was elected leader, sending a full tablet report...
I20260807 05:11:52.180462 23111 raft_consensus.cc:348] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:52.180706 23111 raft_consensus.cc:374] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:11:52.180770 23111 raft_consensus.cc:729] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88d6b5da8aba4197a622da1cbc66023d, State: Initialized, Role: FOLLOWER
I20260807 05:11:52.180917 23109 ts_tablet_manager.cc:1428] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260807 05:11:52.180974 23110 raft_consensus.cc:348] T e77bfe8760214d54bd10d9af7d025190 P e9e3de1593ae48d9b4223fc56c69d5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:52.181178 23111 consensus_queue.cc:260] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:52.181257 23110 raft_consensus.cc:374] T e77bfe8760214d54bd10d9af7d025190 P e9e3de1593ae48d9b4223fc56c69d5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:11:52.181304 23110 raft_consensus.cc:729] T e77bfe8760214d54bd10d9af7d025190 P e9e3de1593ae48d9b4223fc56c69d5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9e3de1593ae48d9b4223fc56c69d5aa, State: Initialized, Role: FOLLOWER
I20260807 05:11:52.181692 23110 consensus_queue.cc:260] T e77bfe8760214d54bd10d9af7d025190 P e9e3de1593ae48d9b4223fc56c69d5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:52.181784 22803 heartbeater.cc:503] Master 127.8.16.126:44131 was elected leader, sending a full tablet report...
I20260807 05:11:52.182163 23111 ts_tablet_manager.cc:1428] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260807 05:11:52.182329 23092 heartbeater.cc:503] Master 127.8.16.126:44131 was elected leader, sending a full tablet report...
I20260807 05:11:52.182593 23110 ts_tablet_manager.cc:1428] T e77bfe8760214d54bd10d9af7d025190 P e9e3de1593ae48d9b4223fc56c69d5aa: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260807 05:11:52.183560 22805 tablet.cc:2369] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:11:52.359513 23115 raft_consensus.cc:482] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:11:52.359773 23115 raft_consensus.cc:504] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:52.360975 23115 leader_election.cc:288] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9e3de1593ae48d9b4223fc56c69d5aa (127.8.16.67:46527), 88d6b5da8aba4197a622da1cbc66023d (127.8.16.65:32949)
W20260807 05:11:52.361135 23093 tablet.cc:2369] T e77bfe8760214d54bd10d9af7d025190 P e9e3de1593ae48d9b4223fc56c69d5aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:11:52.364764 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:52.361630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:56977 (local address 127.8.16.65:32949)
0807 05:11:52.361746 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:52.361750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:52.361768 (+ 18us) server_negotiation.cc:407] Connection header received
0807 05:11:52.361906 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:52.361909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:52.361959 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:52.362027 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:11:52.362287 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.362771 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.363852 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.364241 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.364317 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:52.364380 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:52.364565 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 05:11:52.364637 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 05:11:52.364791 22952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:52.361198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:46527 (local address 127.8.16.66:49951)
0807 05:11:52.361337 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:52.361354 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:52.361473 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:52.361932 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:52.361935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:52.361948 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:11:52.362078 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:52.362085 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.362823 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.362826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:52.363731 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:52.363740 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.364398 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.364401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:52.364504 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:52.364524 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 05:11:52.364597 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 05:11:52.364663 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":61}
I20260807 05:11:52.364807 23118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:52.361264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:32949 (local address 127.8.16.66:56977)
0807 05:11:52.361702 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:52.361714 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:52.361806 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:52.362057 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:52.362060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:52.362076 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:11:52.362185 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:52.362190 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.362874 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.362876 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:52.363729 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:52.363745 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.364347 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.364351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:52.364441 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:52.364465 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 05:11:52.364536 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 05:11:52.364609 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":48,"thread_start_us":89,"threads_started":1}
I20260807 05:11:52.364806 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:52.361628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:49951 (local address 127.8.16.67:46527)
0807 05:11:52.361750 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:52.361754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:52.361770 (+ 16us) server_negotiation.cc:407] Connection header received
0807 05:11:52.361816 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:52.361818 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:52.361860 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:52.361923 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:11:52.362191 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.362716 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.363850 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.364283 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.364385 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:52.364444 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:52.364619 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 05:11:52.364690 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 05:11:52.365397 23043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e77bfe8760214d54bd10d9af7d025190" candidate_uuid: "93ccbf8081324583b623143d78125e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" is_pre_election: true
I20260807 05:11:52.365605 22754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e77bfe8760214d54bd10d9af7d025190" candidate_uuid: "93ccbf8081324583b623143d78125e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88d6b5da8aba4197a622da1cbc66023d" is_pre_election: true
I20260807 05:11:52.365713 23043 raft_consensus.cc:2454] T e77bfe8760214d54bd10d9af7d025190 P e9e3de1593ae48d9b4223fc56c69d5aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93ccbf8081324583b623143d78125e41 in term 0.
I20260807 05:11:52.365881 22754 raft_consensus.cc:2454] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93ccbf8081324583b623143d78125e41 in term 0.
I20260807 05:11:52.366195 22826 leader_election.cc:302] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93ccbf8081324583b623143d78125e41, e9e3de1593ae48d9b4223fc56c69d5aa; no voters:
I20260807 05:11:52.366480 23115 raft_consensus.cc:2781] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:11:52.366557 23115 raft_consensus.cc:482] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:11:52.366586 23115 raft_consensus.cc:3037] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:11:52.367609 23115 raft_consensus.cc:504] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:52.367930 23115 leader_election.cc:288] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [CANDIDATE]: Term 1 election: Requested vote from peers e9e3de1593ae48d9b4223fc56c69d5aa (127.8.16.67:46527), 88d6b5da8aba4197a622da1cbc66023d (127.8.16.65:32949)
I20260807 05:11:52.368239 23043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e77bfe8760214d54bd10d9af7d025190" candidate_uuid: "93ccbf8081324583b623143d78125e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa"
I20260807 05:11:52.368258 22754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e77bfe8760214d54bd10d9af7d025190" candidate_uuid: "93ccbf8081324583b623143d78125e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88d6b5da8aba4197a622da1cbc66023d"
I20260807 05:11:52.368350 23043 raft_consensus.cc:3037] T e77bfe8760214d54bd10d9af7d025190 P e9e3de1593ae48d9b4223fc56c69d5aa [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:11:52.368383 22754 raft_consensus.cc:3037] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:11:52.369366 23043 raft_consensus.cc:2454] T e77bfe8760214d54bd10d9af7d025190 P e9e3de1593ae48d9b4223fc56c69d5aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93ccbf8081324583b623143d78125e41 in term 1.
I20260807 05:11:52.369366 22754 raft_consensus.cc:2454] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93ccbf8081324583b623143d78125e41 in term 1.
I20260807 05:11:52.369659 22827 leader_election.cc:302] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88d6b5da8aba4197a622da1cbc66023d, 93ccbf8081324583b623143d78125e41; no voters:
I20260807 05:11:52.369832 23115 raft_consensus.cc:2781] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:11:52.370048 23115 raft_consensus.cc:686] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [term 1 LEADER]: Becoming Leader. State: Replica: 93ccbf8081324583b623143d78125e41, State: Running, Role: LEADER
I20260807 05:11:52.370397 23115 consensus_queue.cc:237] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:52.373325 22593 catalog_manager.cc:5495] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 93ccbf8081324583b623143d78125e41 (127.8.16.66). New cstate: current_term: 1 leader_uuid: "93ccbf8081324583b623143d78125e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } health_report { overall_health: UNKNOWN } } }
W20260807 05:11:52.393043 22951 tablet.cc:2369] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:11:52.397289 23106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:52.394301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48754 (local address 127.8.16.66:46281)
0807 05:11:52.394420 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:52.394423 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:52.394436 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:11:52.394545 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:52.394547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:52.394600 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:52.394695 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:11:52.394892 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.395378 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.396124 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.396324 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.396386 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:52.396606 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:11:52.396998 (+ 392us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:11:52.397084 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:52.397110 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 05:11:52.397162 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:11:52.402738 22754 raft_consensus.cc:1261] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Refusing update from remote peer 93ccbf8081324583b623143d78125e41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:11:52.402745 23043 raft_consensus.cc:1261] T e77bfe8760214d54bd10d9af7d025190 P e9e3de1593ae48d9b4223fc56c69d5aa [term 1 FOLLOWER]: Refusing update from remote peer 93ccbf8081324583b623143d78125e41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:11:52.403424 23119 consensus_queue.cc:1035] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:11:52.403595 23115 consensus_queue.cc:1035] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:11:52.417562 23124 mvcc.cc:205] Tried to move back new op lower bound from 7315781682794721280 to 7315781682671697920. Current Snapshot: MvccSnapshot[applied={T|T < 7315781682794721280 or (T in {7315781682794721280})}]
I20260807 05:11:52.421728 22593 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:11:52.421886 22593 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:11:52.424546 22593 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e: Sending DeleteTablet for 3 replicas of tablet e77bfe8760214d54bd10d9af7d025190
I20260807 05:11:52.425357 22877 tablet_service.cc:1514] Processing DeleteTablet for tablet e77bfe8760214d54bd10d9af7d025190 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:11:52 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:48746
I20260807 05:11:52.425356 23021 tablet_service.cc:1514] Processing DeleteTablet for tablet e77bfe8760214d54bd10d9af7d025190 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:11:52 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:60118
I20260807 05:11:52.425513 22732 tablet_service.cc:1514] Processing DeleteTablet for tablet e77bfe8760214d54bd10d9af7d025190 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:11:52 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:59570
I20260807 05:11:52.425917 23138 tablet_replica.cc:330] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d: stopping tablet replica
I20260807 05:11:52.426205 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 22660
I20260807 05:11:52.429706 23137 tablet_replica.cc:330] T e77bfe8760214d54bd10d9af7d025190 P e9e3de1593ae48d9b4223fc56c69d5aa: stopping tablet replica
I20260807 05:11:52.429967 23136 tablet_replica.cc:330] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41: stopping tablet replica
I20260807 05:11:52.430315 23136 raft_consensus.cc:2229] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:11:52.430778 23136 raft_consensus.cc:2258] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:11:52.431279 23137 raft_consensus.cc:2229] T e77bfe8760214d54bd10d9af7d025190 P e9e3de1593ae48d9b4223fc56c69d5aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:11:52.431603 23137 raft_consensus.cc:2258] T e77bfe8760214d54bd10d9af7d025190 P e9e3de1593ae48d9b4223fc56c69d5aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:11:52.433360 23136 ts_tablet_manager.cc:1905] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:11:52.433319 23137 ts_tablet_manager.cc:1905] T e77bfe8760214d54bd10d9af7d025190 P e9e3de1593ae48d9b4223fc56c69d5aa: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 05:11:52.433553 22580 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.16.65:32949 (error 108)
I20260807 05:11:52.434453 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 22809
I20260807 05:11:52.436731 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:52.436397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:32949 (local address 127.0.0.1:59586)
0807 05:11:52.436528 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:52.436629 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.65:32949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 05:11:52.436913 22580 catalog_manager.cc:4584] TS 88d6b5da8aba4197a622da1cbc66023d (127.8.16.65:32949): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e77bfe8760214d54bd10d9af7d025190: Network error: Client connection negotiation failed: client connection to 127.8.16.65:32949: connect: Connection refused (error 111)
I20260807 05:11:52.437616 23137 ts_tablet_manager.cc:1918] T e77bfe8760214d54bd10d9af7d025190 P e9e3de1593ae48d9b4223fc56c69d5aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:11:52.437709 23137 log.cc:1196] T e77bfe8760214d54bd10d9af7d025190 P e9e3de1593ae48d9b4223fc56c69d5aa: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/e77bfe8760214d54bd10d9af7d025190
I20260807 05:11:52.437983 23137 ts_tablet_manager.cc:1939] T e77bfe8760214d54bd10d9af7d025190 P e9e3de1593ae48d9b4223fc56c69d5aa: Deleting consensus metadata
I20260807 05:11:52.439112 22579 catalog_manager.cc:4841] TS e9e3de1593ae48d9b4223fc56c69d5aa (127.8.16.67:46527): tablet e77bfe8760214d54bd10d9af7d025190 (table test-table [id=60d7987ad8704365b9e2f2662e0f2290]) successfully deleted
I20260807 05:11:52.440804 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 22955
I20260807 05:11:52.442094 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:52.441709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:46281 (local address 127.0.0.1:48760)
0807 05:11:52.441958 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:52.442012 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.66:46281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
I20260807 05:11:52.446658 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 22561
I20260807 05:11:52.453697 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:44131
--webserver_interface=127.8.16.126
--webserver_port=34171
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:35985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:44131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:11:52.561674 23140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:52.561944 23140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:52.561985 23140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:52.565946 23140 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:11:52.566012 23140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:52.566033 23140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:52.566084 23140 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:11:52.566111 23140 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:11:52.571480 23140 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:35985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:44131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:44131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=34171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:52.572518 23140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:52.573557 23140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:11:52.586349 23149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:52.586375 23147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:52.586313 23146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:52.586484 23140 server_base.cc:1034] running on GCE node
I20260807 05:11:52.587229 23140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:52.588343 23140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:52.589504 23140 hybrid_clock.cc:648] HybridClock initialized: now 1786079512589478 us; error 39 us; skew 500 ppm
Aug 07 05:11:52 dist-test-slave-49h3 krb5kdc[22548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079512, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:52.592088 23140 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:11:52.592927 23140 webserver.cc:460] Webserver started at http://127.8.16.126:34171/ using document root <none> and password file <none>
I20260807 05:11:52.593437 23140 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:52.593554 23140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:11:52.597148 23140 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260807 05:11:52.599191 23157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:52.600358 23140 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:11:52.600476 23140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "6c5c1de2736e4027a7a6b15d06e19a1e"
format_stamp: "Formatted at 2026-08-07 05:11:51 on dist-test-slave-49h3"
server_key: "442cc7ac7135d34c381312558e5c6841"
server_key_iv: "7d0873c9f1275e5b520cce454bc753b2"
server_key_version: "encryptionkey@0"
I20260807 05:11:52.601059 23140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:52.643471 23140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:52.646842 23140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:52.647040 23140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:52.653959 23140 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:44131
I20260807 05:11:52.654210 23213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:44131 every 8 connection(s)
I20260807 05:11:52.655058 23140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:11:52.659350 23214 sys_catalog.cc:263] Verifying existing consensus state
I20260807 05:11:52.659858 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 23140
I20260807 05:11:52.660354 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:32949
--local_ip_for_outbound_sockets=127.8.16.65
--tserver_master_addrs=127.8.16.126:44131
--webserver_port=42171
--webserver_interface=127.8.16.65
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:35985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:11:52.661402 23214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e: Bootstrap starting.
I20260807 05:11:52.678429 23214 log.cc:824] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e: Log is configured to *not* fsync() on all Append() calls
I20260807 05:11:52.687027 23214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:11:52.687386 23214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e: Bootstrap complete.
I20260807 05:11:52.691339 23214 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44131 } }
I20260807 05:11:52.691756 23214 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c5c1de2736e4027a7a6b15d06e19a1e, State: Initialized, Role: FOLLOWER
I20260807 05:11:52.692247 23214 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44131 } }
I20260807 05:11:52.692368 23214 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:11:52.692435 23214 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:11:52.692523 23214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:11:52.694482 23214 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44131 } }
I20260807 05:11:52.694805 23214 leader_election.cc:302] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c5c1de2736e4027a7a6b15d06e19a1e; no voters:
I20260807 05:11:52.695111 23214 leader_election.cc:288] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 05:11:52.695253 23218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 2 FOLLOWER]: Leader election won for term 2
I20260807 05:11:52.695461 23218 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 2 LEADER]: Becoming Leader. State: Replica: 6c5c1de2736e4027a7a6b15d06e19a1e, State: Running, Role: LEADER
I20260807 05:11:52.695811 23218 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44131 } }
I20260807 05:11:52.696363 23214 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:11:52.697250 23219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44131 } } }
I20260807 05:11:52.697391 23219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [sys.catalog]: This master's current role is: LEADER
I20260807 05:11:52.697703 23220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c5c1de2736e4027a7a6b15d06e19a1e. Latest consensus state: current_term: 2 leader_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44131 } } }
I20260807 05:11:52.697784 23227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:11:52.697791 23220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [sys.catalog]: This master's current role is: LEADER
I20260807 05:11:52.702463 23227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:11:52.703164 23227 catalog_manager.cc:1259] Loaded cluster ID: e9b118e990054a0aa4e104debe822d61
I20260807 05:11:52.703222 23227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:11:52.704367 23227 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 05:11:52.707469 23227 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 05:11:52.711941 23227 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:11:52.712651 23227 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e: Loaded TSK: 0
I20260807 05:11:52.713621 23227 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 05:11:52.786405 23216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:52.786661 23216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:52.786721 23216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:52.790614 23216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:52.790671 23216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:52.790748 23216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:11:52.795825 23216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:35985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:32949
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=42171
--tserver_master_addrs=127.8.16.126:44131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:52.796851 23216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:52.797901 23216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:11:52.807727 23243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:52.807904 23246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:52.807965 23216 server_base.cc:1034] running on GCE node
W20260807 05:11:52.808100 23244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:52.808568 23216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:52.809306 23216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:52.810506 23216 hybrid_clock.cc:648] HybridClock initialized: now 1786079512810493 us; error 44 us; skew 500 ppm
Aug 07 05:11:52 dist-test-slave-49h3 krb5kdc[22548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079512, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:52.813840 23216 init.cc:377] Logged in from keytab as kudu/127.8.16.65@KRBTEST.COM (short username kudu)
I20260807 05:11:52.814915 23216 webserver.cc:460] Webserver started at http://127.8.16.65:42171/ using document root <none> and password file <none>
I20260807 05:11:52.815606 23216 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:52.815722 23216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:11:52.820850 23216 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:11:52.823508 23253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:52.824978 23216 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:11:52.825107 23216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "88d6b5da8aba4197a622da1cbc66023d"
format_stamp: "Formatted at 2026-08-07 05:11:51 on dist-test-slave-49h3"
server_key: "12c85cb5b76908b1f9af9f48d53c4897"
server_key_iv: "bf9fc055a5ab96ec809a178d9416b7b2"
server_key_version: "encryptionkey@0"
I20260807 05:11:52.825806 23216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:52.857756 23216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:52.861559 23216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:52.861825 23216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:52.862741 23216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:11:52.864250 23260 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 05:11:52.870381 23216 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 05:11:52.870491 23216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260807 05:11:52.870586 23216 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 05:11:52.873044 23216 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 05:11:52.873117 23216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260807 05:11:52.873566 23260 tablet_bootstrap.cc:492] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d: Bootstrap starting.
I20260807 05:11:52.888177 23216 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:32949
I20260807 05:11:52.888311 23376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:32949 every 8 connection(s)
I20260807 05:11:52.889556 23216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:11:52.898268 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 23216
I20260807 05:11:52.899019 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:46281
--local_ip_for_outbound_sockets=127.8.16.66
--tserver_master_addrs=127.8.16.126:44131
--webserver_port=46183
--webserver_interface=127.8.16.66
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:35985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:11:52.899212 23260 log.cc:824] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d: Log is configured to *not* fsync() on all Append() calls
Aug 07 05:11:52 dist-test-slave-49h3 krb5kdc[22548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079512, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:11:52.912796 23260 tablet_bootstrap.cc:492] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:11:52.913489 23260 tablet_bootstrap.cc:492] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d: Bootstrap complete.
I20260807 05:11:52.914812 23379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:52.892241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.65:40351)
0807 05:11:52.893091 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:52.893185 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:52.895010 (+ 1825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:52.896629 (+ 1619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:52.896651 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:52.897502 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:52.898578 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:52.898614 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.900950 (+ 2336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.900959 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:52.902505 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:52.902515 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.903550 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.903562 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:52.903873 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:52.904700 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:52.904725 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:52.907061 (+ 2336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:52.910171 (+ 3110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:52.910184 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:52.910203 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:52.910584 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:52.911051 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:52.911054 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:52.911056 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:52.911247 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:52.912016 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:52.912029 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:52.912397 (+ 368us) client_negotiation.cc:769] Sending connection context
0807 05:11:52.912621 (+ 224us) client_negotiation.cc:240] Negotiation successful
0807 05:11:52.913272 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":218,"threads_started":1}
I20260807 05:11:52.914793 23380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:52.892455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:40351 (local address 127.8.16.126:44131)
0807 05:11:52.893167 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:52.893204 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:52.893342 (+ 138us) server_negotiation.cc:407] Connection header received
0807 05:11:52.895637 (+ 2295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:52.895659 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:52.896051 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:52.896455 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:52.898914 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.900720 (+ 1806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.902662 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.903363 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.903618 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:52.907340 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:52.907355 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:52.907374 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:52.907418 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:52.910013 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:52.910739 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:52.910746 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:52.910750 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:52.910920 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:52.911373 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:52.911375 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:52.911377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:52.911862 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:52.912029 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:52.912840 (+ 811us) server_negotiation.cc:299] Negotiation successful
0807 05:11:52.913272 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":166,"threads_started":1}
I20260807 05:11:52.915247 23260 ts_tablet_manager.cc:1397] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d: Time spent bootstrapping tablet: real 0.042s user 0.028s sys 0.011s
I20260807 05:11:52.918375 23377 heartbeater.cc:347] Connected to a master server at 127.8.16.126:44131
I20260807 05:11:52.918677 23377 heartbeater.cc:464] Registering TS with master...
I20260807 05:11:52.919240 23377 heartbeater.cc:511] Master 127.8.16.126:44131 requested a full tablet report, sending...
I20260807 05:11:52.919981 23260 raft_consensus.cc:348] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:52.920789 23260 raft_consensus.cc:729] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88d6b5da8aba4197a622da1cbc66023d, State: Initialized, Role: FOLLOWER
I20260807 05:11:52.921617 23260 consensus_queue.cc:260] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:52.922480 23176 ts_manager.cc:194] Registered new tserver with Master: 88d6b5da8aba4197a622da1cbc66023d (127.8.16.65:32949)
I20260807 05:11:52.923238 23260 ts_tablet_manager.cc:1428] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260807 05:11:52.927973 23176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:40351
I20260807 05:11:52.932523 23377 heartbeater.cc:503] Master 127.8.16.126:44131 was elected leader, sending a full tablet report...
Aug 07 05:11:52 dist-test-slave-49h3 krb5kdc[22548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079512, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for kudu/127.8.16.65@KRBTEST.COM
I20260807 05:11:52.941874 23385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:52.926652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:32949 (local address 127.0.0.1:59602)
0807 05:11:52.927574 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:52.927659 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:52.927894 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:52.928722 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:52.928731 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:52.929557 (+ 826us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:52.929744 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:52.929754 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.931139 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.931144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:52.932080 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:52.932095 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.933004 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.933015 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:52.933505 (+ 490us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:52.934486 (+ 981us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:52.934517 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:52.936601 (+ 2084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:52.939343 (+ 2742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:52.939360 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:52.939366 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:52.939740 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:52.940119 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:52.940122 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:52.940125 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:52.940238 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:52.940858 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:52.940872 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:52.941048 (+ 176us) client_negotiation.cc:769] Sending connection context
0807 05:11:52.941258 (+ 210us) client_negotiation.cc:240] Negotiation successful
0807 05:11:52.941599 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":186,"threads_started":1}
I20260807 05:11:52.941906 23384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:52.926724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59602 (local address 127.8.16.65:32949)
0807 05:11:52.927138 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:52.927162 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:52.927810 (+ 648us) server_negotiation.cc:407] Connection header received
0807 05:11:52.928134 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:52.928150 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:52.928489 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:52.928711 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:52.929890 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.931000 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.932304 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:52.932850 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:52.933003 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:52.936792 (+ 3789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:52.936801 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:52.936817 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:52.936850 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:52.939208 (+ 2358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:52.939881 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:52.939888 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:52.939889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:52.939991 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:52.940351 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:52.940354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:52.940356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:52.940679 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:52.940871 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:52.941389 (+ 518us) server_negotiation.cc:299] Negotiation successful
0807 05:11:52.941660 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":267,"thread_start_us":132,"threads_started":1}
I20260807 05:11:52.943238 23307 tablet_service.cc:1514] Processing DeleteTablet for tablet e77bfe8760214d54bd10d9af7d025190 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:11:52 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:59602
I20260807 05:11:52.944320 23386 tablet_replica.cc:330] stopping tablet replica
I20260807 05:11:52.944717 23386 raft_consensus.cc:2229] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:11:52.945060 23386 raft_consensus.cc:2258] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:11:52.946421 23386 ts_tablet_manager.cc:1905] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:11:52.951690 23386 ts_tablet_manager.cc:1918] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:11:52.951831 23386 log.cc:1196] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/wal/wals/e77bfe8760214d54bd10d9af7d025190
I20260807 05:11:52.952219 23386 ts_tablet_manager.cc:1939] T e77bfe8760214d54bd10d9af7d025190 P 88d6b5da8aba4197a622da1cbc66023d: Deleting consensus metadata
I20260807 05:11:52.954623 23161 catalog_manager.cc:4841] TS 88d6b5da8aba4197a622da1cbc66023d (127.8.16.65:32949): tablet e77bfe8760214d54bd10d9af7d025190 (table test-table [id=60d7987ad8704365b9e2f2662e0f2290]) successfully deleted
W20260807 05:11:53.082660 23382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:53.083106 23382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:53.083214 23382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:53.088871 23382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:53.088975 23382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:53.089123 23382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:11:53.096748 23382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:35985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:46281
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=46183
--tserver_master_addrs=127.8.16.126:44131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:53.098511 23382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:53.100054 23382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:11:53.114156 23395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:53.114395 23382 server_base.cc:1034] running on GCE node
W20260807 05:11:53.114226 23393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:53.114184 23391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:53.115063 23382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:53.115921 23382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:53.117120 23382 hybrid_clock.cc:648] HybridClock initialized: now 1786079513117081 us; error 58 us; skew 500 ppm
Aug 07 05:11:53 dist-test-slave-49h3 krb5kdc[22548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079513, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:53.121069 23382 init.cc:377] Logged in from keytab as kudu/127.8.16.66@KRBTEST.COM (short username kudu)
I20260807 05:11:53.122396 23382 webserver.cc:460] Webserver started at http://127.8.16.66:46183/ using document root <none> and password file <none>
I20260807 05:11:53.123181 23382 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:53.123320 23382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:11:53.129426 23382 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260807 05:11:53.132891 23402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:53.134876 23382 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260807 05:11:53.135048 23382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "93ccbf8081324583b623143d78125e41"
format_stamp: "Formatted at 2026-08-07 05:11:51 on dist-test-slave-49h3"
server_key: "12b862660c003cdbc8804c2742ca4025"
server_key_iv: "99a040659bf31996dcf05def6e548ecb"
server_key_version: "encryptionkey@0"
I20260807 05:11:53.135890 23382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:53.172958 23382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:53.177990 23382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:53.178301 23382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:53.179343 23382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:11:53.181066 23409 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 05:11:53.187722 23382 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 05:11:53.187841 23382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260807 05:11:53.187981 23382 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 05:11:53.190660 23382 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 05:11:53.190738 23382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20260807 05:11:53.191114 23409 tablet_bootstrap.cc:492] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41: Bootstrap starting.
I20260807 05:11:53.200138 23409 log.cc:824] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41: Log is configured to *not* fsync() on all Append() calls
I20260807 05:11:53.207890 23382 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:46281
I20260807 05:11:53.208033 23525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:46281 every 8 connection(s)
I20260807 05:11:53.209331 23382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:11:53.212831 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 23382
I20260807 05:11:53.213717 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:46527
--local_ip_for_outbound_sockets=127.8.16.67
--tserver_master_addrs=127.8.16.126:44131
--webserver_port=39691
--webserver_interface=127.8.16.67
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:35985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:11:53.214777 23409 tablet_bootstrap.cc:492] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:11:53.215523 23409 tablet_bootstrap.cc:492] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41: Bootstrap complete.
I20260807 05:11:53.216111 23409 ts_tablet_manager.cc:1397] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41: Time spent bootstrapping tablet: real 0.025s user 0.022s sys 0.000s
I20260807 05:11:53.220129 23409 raft_consensus.cc:348] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:53.221093 23409 raft_consensus.cc:729] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93ccbf8081324583b623143d78125e41, State: Initialized, Role: FOLLOWER
I20260807 05:11:53.222009 23409 consensus_queue.cc:260] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:53.224009 23409 ts_tablet_manager.cc:1428] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
Aug 07 05:11:53 dist-test-slave-49h3 krb5kdc[22548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079513, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:11:53.232993 23380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:53.212500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:47953 (local address 127.8.16.126:44131)
0807 05:11:53.212907 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:53.212924 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:53.213914 (+ 990us) server_negotiation.cc:407] Connection header received
0807 05:11:53.215736 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:53.215740 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:53.215837 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:53.215991 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:53.218847 (+ 2856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.219560 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.221346 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.221837 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.222004 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:53.226567 (+ 4563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:53.226572 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:53.226574 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:53.226624 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:53.229213 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:53.230028 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:53.230038 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:53.230041 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:53.230184 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:53.230755 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:53.230762 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:53.230764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:53.231320 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:53.231525 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:53.232377 (+ 852us) server_negotiation.cc:299] Negotiation successful
0807 05:11:53.232658 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264}
I20260807 05:11:53.234268 23528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:53.212761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.66:47953)
0807 05:11:53.213663 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:53.213753 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:53.215448 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:53.216497 (+ 1049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:53.216515 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:53.217402 (+ 887us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:53.218572 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:53.218602 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.219780 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.219783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:53.221171 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:53.221189 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.222005 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.222010 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:53.222381 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:53.223816 (+ 1435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:53.223846 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:53.226309 (+ 2463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:53.229371 (+ 3062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:53.229388 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:53.229409 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:53.229881 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:53.230348 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:53.230351 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:53.230352 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:53.230572 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:53.231565 (+ 993us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:53.231572 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:53.231960 (+ 388us) client_negotiation.cc:769] Sending connection context
0807 05:11:53.232227 (+ 267us) client_negotiation.cc:240] Negotiation successful
0807 05:11:53.232653 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":231,"threads_started":1}
I20260807 05:11:53.236665 23526 heartbeater.cc:347] Connected to a master server at 127.8.16.126:44131
I20260807 05:11:53.236987 23526 heartbeater.cc:464] Registering TS with master...
I20260807 05:11:53.237776 23526 heartbeater.cc:511] Master 127.8.16.126:44131 requested a full tablet report, sending...
I20260807 05:11:53.240118 23176 ts_manager.cc:194] Registered new tserver with Master: 93ccbf8081324583b623143d78125e41 (127.8.16.66:46281)
I20260807 05:11:53.241808 23176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:47953
I20260807 05:11:53.245419 23526 heartbeater.cc:503] Master 127.8.16.126:44131 was elected leader, sending a full tablet report...
Aug 07 05:11:53 dist-test-slave-49h3 krb5kdc[22548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079512, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for kudu/127.8.16.66@KRBTEST.COM
I20260807 05:11:53.257185 23385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:53.241406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:46281 (local address 127.0.0.1:48770)
0807 05:11:53.241768 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:53.241792 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:53.241929 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:53.242963 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:53.242967 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:53.243301 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:53.243498 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:53.243516 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.245546 (+ 2030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.245551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:53.246818 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:53.246837 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.247968 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.247978 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:53.248165 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:53.248907 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:53.248934 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:53.250936 (+ 2002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:53.254668 (+ 3732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:53.254676 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:53.254678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:53.254979 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:53.255471 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:53.255473 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:53.255474 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:53.255587 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:53.256503 (+ 916us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:53.256511 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:53.256664 (+ 153us) client_negotiation.cc:769] Sending connection context
0807 05:11:53.256803 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 05:11:53.257010 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241}
I20260807 05:11:53.257589 23531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:53.241576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48770 (local address 127.8.16.66:46281)
0807 05:11:53.242071 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:53.242089 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:53.242126 (+ 37us) server_negotiation.cc:407] Connection header received
0807 05:11:53.242440 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:53.242450 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:53.242702 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:53.242987 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:53.243723 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.245305 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.247007 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.247796 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.247914 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:53.251170 (+ 3256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:53.251189 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:53.251209 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:53.251251 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:53.254485 (+ 3234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:53.255148 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:53.255164 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:53.255166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:53.255321 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:53.255724 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:53.255731 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:53.255740 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:53.256317 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:53.256575 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:53.256975 (+ 400us) server_negotiation.cc:299] Negotiation successful
0807 05:11:53.257273 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":343,"thread_start_us":176,"threads_started":1}
I20260807 05:11:53.259147 23456 tablet_service.cc:1514] Processing DeleteTablet for tablet e77bfe8760214d54bd10d9af7d025190 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:11:52 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:48770
I20260807 05:11:53.260397 23532 tablet_replica.cc:330] stopping tablet replica
I20260807 05:11:53.260849 23532 raft_consensus.cc:2229] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:11:53.261183 23532 raft_consensus.cc:2258] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:11:53.262660 23532 ts_tablet_manager.cc:1905] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:11:53.268258 23532 ts_tablet_manager.cc:1918] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:11:53.268404 23532 log.cc:1196] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/wal/wals/e77bfe8760214d54bd10d9af7d025190
I20260807 05:11:53.268805 23532 ts_tablet_manager.cc:1939] T e77bfe8760214d54bd10d9af7d025190 P 93ccbf8081324583b623143d78125e41: Deleting consensus metadata
I20260807 05:11:53.271091 23161 catalog_manager.cc:4841] TS 93ccbf8081324583b623143d78125e41 (127.8.16.66:46281): tablet e77bfe8760214d54bd10d9af7d025190 (table test-table [id=60d7987ad8704365b9e2f2662e0f2290]) successfully deleted
W20260807 05:11:53.387496 23529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:53.387897 23529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:53.387995 23529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:53.393527 23529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:53.393630 23529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:53.393777 23529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:11:53.401072 23529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:35985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:46527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=39691
--tserver_master_addrs=127.8.16.126:44131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:53.402601 23529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:53.404081 23529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:11:53.416962 23538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:53.416962 23540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:53.417001 23537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:53.417409 23529 server_base.cc:1034] running on GCE node
I20260807 05:11:53.418006 23529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:53.418850 23529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:53.420044 23529 hybrid_clock.cc:648] HybridClock initialized: now 1786079513420011 us; error 49 us; skew 500 ppm
Aug 07 05:11:53 dist-test-slave-49h3 krb5kdc[22548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079513, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:53.423867 23529 init.cc:377] Logged in from keytab as kudu/127.8.16.67@KRBTEST.COM (short username kudu)
I20260807 05:11:53.425036 23529 webserver.cc:460] Webserver started at http://127.8.16.67:39691/ using document root <none> and password file <none>
I20260807 05:11:53.425875 23529 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:53.425999 23529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:11:53.431453 23529 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 05:11:53.434262 23547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:53.435958 23529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:11:53.436101 23529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "e9e3de1593ae48d9b4223fc56c69d5aa"
format_stamp: "Formatted at 2026-08-07 05:11:52 on dist-test-slave-49h3"
server_key: "42dd5bb737a9c6f9196ce5b08ebfd628"
server_key_iv: "8c8d414f2c91c77e94a024034b3312cb"
server_key_version: "encryptionkey@0"
I20260807 05:11:53.436823 23529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:53.453979 23529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:53.458105 23529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:53.458369 23529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:53.459185 23529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:11:53.460359 23529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:11:53.460429 23529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:53.460506 23529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:11:53.460551 23529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:53.474895 23529 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:46527
I20260807 05:11:53.474956 23674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:46527 every 8 connection(s)
I20260807 05:11:53.476619 23529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:11:53.486847 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 23529
Aug 07 05:11:53 dist-test-slave-49h3 krb5kdc[22548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079513, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:11:53.505034 23678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:53.488217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45768 (local address 127.8.16.126:44131)
0807 05:11:53.488741 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:53.488746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:53.488945 (+ 199us) server_negotiation.cc:407] Connection header received
0807 05:11:53.489246 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:53.489262 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:53.489347 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:53.489614 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:53.490704 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.492107 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.493851 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.494725 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.495116 (+ 391us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:53.497975 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:53.497980 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:53.497983 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:53.498069 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:53.501659 (+ 3590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:53.502452 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:53.502466 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:53.502478 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:53.502634 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:53.503026 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:53.503040 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:53.503041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:53.503699 (+ 658us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:53.504082 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:53.504432 (+ 350us) server_negotiation.cc:299] Negotiation successful
0807 05:11:53.504683 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":155,"threads_started":1}
I20260807 05:11:53.506304 23380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:53.479696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:39435 (local address 127.8.16.126:44131)
0807 05:11:53.479928 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:53.479938 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:53.482337 (+ 2399us) server_negotiation.cc:407] Connection header received
0807 05:11:53.484312 (+ 1975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:53.484323 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:53.484395 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:53.484547 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:53.487013 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.488189 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.490869 (+ 2680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.491456 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.491644 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:53.497844 (+ 6200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:53.497848 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:53.497851 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:53.497884 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:53.501277 (+ 3393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:53.502403 (+ 1126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:53.502407 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:53.502408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:53.502603 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:53.503490 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:53.503494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:53.503505 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:53.503976 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:53.504173 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:53.505605 (+ 1432us) server_negotiation.cc:299] Negotiation successful
0807 05:11:53.505821 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"spinlock_wait_cycles":28032}
I20260807 05:11:53.508249 23677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:53.480428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.67:39435)
0807 05:11:53.481989 (+ 1561us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:53.482122 (+ 133us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:53.483858 (+ 1736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:53.484872 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:53.484888 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:53.485535 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:53.486590 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:53.486639 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.488417 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.488432 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:53.490589 (+ 2157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:53.490620 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.491903 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.491917 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:53.492467 (+ 550us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:53.493580 (+ 1113us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:53.493632 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:53.497431 (+ 3799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:53.501584 (+ 4153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:53.501611 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:53.501649 (+ 38us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:53.502235 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:53.502780 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:53.502785 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:53.502796 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:53.503154 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:53.504235 (+ 1081us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:53.504253 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:53.504784 (+ 531us) client_negotiation.cc:769] Sending connection context
0807 05:11:53.505224 (+ 440us) client_negotiation.cc:240] Negotiation successful
0807 05:11:53.506021 (+ 797us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1115,"thread_start_us":360,"threads_started":1}
I20260807 05:11:53.510202 23675 heartbeater.cc:347] Connected to a master server at 127.8.16.126:44131
I20260807 05:11:53.510628 23675 heartbeater.cc:464] Registering TS with master...
I20260807 05:11:53.511472 23675 heartbeater.cc:511] Master 127.8.16.126:44131 requested a full tablet report, sending...
I20260807 05:11:53.513167 23176 ts_manager.cc:194] Registered new tserver with Master: e9e3de1593ae48d9b4223fc56c69d5aa (127.8.16.67:46527)
I20260807 05:11:53.514454 23176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:39435
I20260807 05:11:53.521000 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:11:53.526818 23380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:53.521756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45772 (local address 127.8.16.126:44131)
0807 05:11:53.521941 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:53.521948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:53.522190 (+ 242us) server_negotiation.cc:407] Connection header received
0807 05:11:53.522392 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:53.522394 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:53.522554 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:53.522669 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:11:53.522932 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.523557 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.524654 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.524970 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.525043 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:53.525328 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:11:53.526146 (+ 818us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:11:53.526309 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:53.526420 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 05:11:53.526517 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 05:11:53.527967 23176 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:45772:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:11:53.535398 23176 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:11:53.546669 23456 tablet_service.cc:1467] Processing CreateTablet for tablet 081fceaa0d954a58a3ffcf1c31b953c6 (DEFAULT_TABLE table=test-table [id=e22c1a8b310d43588b56b048cf6bb5e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:11:53.546787 23307 tablet_service.cc:1467] Processing CreateTablet for tablet 081fceaa0d954a58a3ffcf1c31b953c6 (DEFAULT_TABLE table=test-table [id=e22c1a8b310d43588b56b048cf6bb5e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:11:53.547600 23456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 081fceaa0d954a58a3ffcf1c31b953c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:11:53.547655 23307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 081fceaa0d954a58a3ffcf1c31b953c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:11:53.551422 23686 tablet_bootstrap.cc:492] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d: Bootstrap starting.
I20260807 05:11:53.552606 23409 tablet_bootstrap.cc:492] T 081fceaa0d954a58a3ffcf1c31b953c6 P 93ccbf8081324583b623143d78125e41: Bootstrap starting.
I20260807 05:11:53.554360 23686 tablet_bootstrap.cc:654] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d: Neither blocks nor log segments found. Creating new log.
I20260807 05:11:53.554427 23409 tablet_bootstrap.cc:654] T 081fceaa0d954a58a3ffcf1c31b953c6 P 93ccbf8081324583b623143d78125e41: Neither blocks nor log segments found. Creating new log.
I20260807 05:11:53.556136 23385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:53.545640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:46527 (local address 127.0.0.1:60134)
0807 05:11:53.545864 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:53.545891 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:53.546006 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:53.547338 (+ 1332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:53.547341 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:53.547372 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:11:53.547531 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:53.547546 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.549765 (+ 2219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.549769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:53.551988 (+ 2219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:53.552015 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.555364 (+ 3349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.555369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:53.555521 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:53.555622 (+ 101us) client_negotiation.cc:769] Sending connection context
0807 05:11:53.555724 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 05:11:53.555879 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118}
I20260807 05:11:53.556618 23686 tablet_bootstrap.cc:492] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d: No bootstrap required, opened a new log
I20260807 05:11:53.556809 23686 ts_tablet_manager.cc:1397] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:11:53.556824 23409 tablet_bootstrap.cc:492] T 081fceaa0d954a58a3ffcf1c31b953c6 P 93ccbf8081324583b623143d78125e41: No bootstrap required, opened a new log
I20260807 05:11:53.556993 23409 ts_tablet_manager.cc:1397] T 081fceaa0d954a58a3ffcf1c31b953c6 P 93ccbf8081324583b623143d78125e41: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260807 05:11:53.557807 23685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:53.545872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60134 (local address 127.8.16.67:46527)
0807 05:11:53.546462 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:53.546486 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:53.546524 (+ 38us) server_negotiation.cc:407] Connection header received
0807 05:11:53.546638 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:53.546644 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:53.547032 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:53.547342 (+ 310us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:11:53.547765 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.549523 (+ 1758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.552917 (+ 3394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.555127 (+ 2210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.556001 (+ 874us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:53.556665 (+ 664us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:53.557298 (+ 633us) server_negotiation.cc:299] Negotiation successful
0807 05:11:53.557527 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":198,"threads_started":1}
I20260807 05:11:53.557696 23686 raft_consensus.cc:348] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:53.557917 23686 raft_consensus.cc:374] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:11:53.558002 23686 raft_consensus.cc:729] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88d6b5da8aba4197a622da1cbc66023d, State: Initialized, Role: FOLLOWER
I20260807 05:11:53.557871 23409 raft_consensus.cc:348] T 081fceaa0d954a58a3ffcf1c31b953c6 P 93ccbf8081324583b623143d78125e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:53.558092 23409 raft_consensus.cc:374] T 081fceaa0d954a58a3ffcf1c31b953c6 P 93ccbf8081324583b623143d78125e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:11:53.558224 23409 raft_consensus.cc:729] T 081fceaa0d954a58a3ffcf1c31b953c6 P 93ccbf8081324583b623143d78125e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93ccbf8081324583b623143d78125e41, State: Initialized, Role: FOLLOWER
I20260807 05:11:53.558248 23686 consensus_queue.cc:260] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:53.558461 23409 consensus_queue.cc:260] T 081fceaa0d954a58a3ffcf1c31b953c6 P 93ccbf8081324583b623143d78125e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:53.558826 23686 ts_tablet_manager.cc:1428] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 05:11:53.559723 23409 ts_tablet_manager.cc:1428] T 081fceaa0d954a58a3ffcf1c31b953c6 P 93ccbf8081324583b623143d78125e41: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 05:11:53.562876 23601 tablet_service.cc:1467] Processing CreateTablet for tablet 081fceaa0d954a58a3ffcf1c31b953c6 (DEFAULT_TABLE table=test-table [id=e22c1a8b310d43588b56b048cf6bb5e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:11:53.564620 23601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 081fceaa0d954a58a3ffcf1c31b953c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:11:53.575210 23690 tablet_bootstrap.cc:492] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa: Bootstrap starting.
I20260807 05:11:53.578438 23690 tablet_bootstrap.cc:654] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa: Neither blocks nor log segments found. Creating new log.
I20260807 05:11:53.580482 23690 log.cc:824] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa: Log is configured to *not* fsync() on all Append() calls
I20260807 05:11:53.583581 23690 tablet_bootstrap.cc:492] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa: No bootstrap required, opened a new log
I20260807 05:11:53.583837 23690 ts_tablet_manager.cc:1397] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 05:11:53.588735 23690 raft_consensus.cc:348] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:53.589311 23690 raft_consensus.cc:374] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:11:53.589422 23690 raft_consensus.cc:729] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9e3de1593ae48d9b4223fc56c69d5aa, State: Initialized, Role: FOLLOWER
I20260807 05:11:53.590441 23690 consensus_queue.cc:260] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:53.590737 23692 raft_consensus.cc:482] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:11:53.591002 23692 raft_consensus.cc:504] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:53.591542 23675 heartbeater.cc:503] Master 127.8.16.126:44131 was elected leader, sending a full tablet report...
I20260807 05:11:53.592332 23690 ts_tablet_manager.cc:1428] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa: Time spent starting tablet: real 0.008s user 0.004s sys 0.005s
I20260807 05:11:53.592970 23692 leader_election.cc:288] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93ccbf8081324583b623143d78125e41 (127.8.16.66:46281), 88d6b5da8aba4197a622da1cbc66023d (127.8.16.65:32949)
I20260807 05:11:53.599831 23677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:53.593267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:46281 (local address 127.8.16.67:41335)
0807 05:11:53.593438 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:53.593491 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:53.593612 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:53.594550 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:53.594553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:53.594571 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:11:53.594788 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:53.594797 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.596364 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.596369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:53.598015 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:53.598032 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.599127 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.599132 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:53.599299 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:53.599337 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 05:11:53.599463 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 05:11:53.599587 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":10}
I20260807 05:11:53.600165 23694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:53.593706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:32949 (local address 127.8.16.67:39973)
0807 05:11:53.594065 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:53.594091 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:53.594235 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:53.595116 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:53.595120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:53.595150 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:11:53.595346 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:53.595364 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.596739 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.596742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:53.598200 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:53.598217 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.599414 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.599429 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:53.599600 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:53.599648 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 05:11:53.599809 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 05:11:53.599953 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":144,"threads_started":1}
I20260807 05:11:53.600183 23695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:53.594323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:39973 (local address 127.8.16.65:32949)
0807 05:11:53.594653 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:53.594673 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:53.594694 (+ 21us) server_negotiation.cc:407] Connection header received
0807 05:11:53.594771 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:53.594773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:53.594846 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:53.595090 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:11:53.595529 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.596625 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.598384 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.599221 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.599399 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:53.599669 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:53.599834 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 05:11:53.599986 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":115,"threads_started":1}
I20260807 05:11:53.600436 23531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:53.593607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:41335 (local address 127.8.16.66:46281)
0807 05:11:53.593818 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:53.593822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:53.593850 (+ 28us) server_negotiation.cc:407] Connection header received
0807 05:11:53.593930 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:53.593944 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:53.594087 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:53.594282 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:11:53.594987 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.596167 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.598199 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.598964 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.599098 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:53.599288 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:53.600096 (+ 808us) server_negotiation.cc:299] Negotiation successful
0807 05:11:53.600224 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 05:11:53.601068 23328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "081fceaa0d954a58a3ffcf1c31b953c6" candidate_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88d6b5da8aba4197a622da1cbc66023d" is_pre_election: true
I20260807 05:11:53.601198 23478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "081fceaa0d954a58a3ffcf1c31b953c6" candidate_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93ccbf8081324583b623143d78125e41" is_pre_election: true
I20260807 05:11:53.601614 23328 raft_consensus.cc:2454] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9e3de1593ae48d9b4223fc56c69d5aa in term 0.
I20260807 05:11:53.601760 23478 raft_consensus.cc:2454] T 081fceaa0d954a58a3ffcf1c31b953c6 P 93ccbf8081324583b623143d78125e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9e3de1593ae48d9b4223fc56c69d5aa in term 0.
I20260807 05:11:53.602375 23550 leader_election.cc:302] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93ccbf8081324583b623143d78125e41, e9e3de1593ae48d9b4223fc56c69d5aa; no voters:
I20260807 05:11:53.602720 23692 raft_consensus.cc:2781] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:11:53.602873 23692 raft_consensus.cc:482] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:11:53.602984 23692 raft_consensus.cc:3037] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:11:53.604667 23692 raft_consensus.cc:504] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:53.605254 23692 leader_election.cc:288] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa [CANDIDATE]: Term 1 election: Requested vote from peers 93ccbf8081324583b623143d78125e41 (127.8.16.66:46281), 88d6b5da8aba4197a622da1cbc66023d (127.8.16.65:32949)
I20260807 05:11:53.605739 23478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "081fceaa0d954a58a3ffcf1c31b953c6" candidate_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93ccbf8081324583b623143d78125e41"
I20260807 05:11:53.605929 23328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "081fceaa0d954a58a3ffcf1c31b953c6" candidate_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88d6b5da8aba4197a622da1cbc66023d"
I20260807 05:11:53.605988 23478 raft_consensus.cc:3037] T 081fceaa0d954a58a3ffcf1c31b953c6 P 93ccbf8081324583b623143d78125e41 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:11:53.606160 23328 raft_consensus.cc:3037] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:11:53.608059 23328 raft_consensus.cc:2454] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9e3de1593ae48d9b4223fc56c69d5aa in term 1.
I20260807 05:11:53.608085 23478 raft_consensus.cc:2454] T 081fceaa0d954a58a3ffcf1c31b953c6 P 93ccbf8081324583b623143d78125e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9e3de1593ae48d9b4223fc56c69d5aa in term 1.
I20260807 05:11:53.608515 23550 leader_election.cc:302] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88d6b5da8aba4197a622da1cbc66023d, e9e3de1593ae48d9b4223fc56c69d5aa; no voters:
I20260807 05:11:53.608734 23692 raft_consensus.cc:2781] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:11:53.608893 23692 raft_consensus.cc:686] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa [term 1 LEADER]: Becoming Leader. State: Replica: e9e3de1593ae48d9b4223fc56c69d5aa, State: Running, Role: LEADER
I20260807 05:11:53.609459 23692 consensus_queue.cc:237] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:53.614576 23175 catalog_manager.cc:5495] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa reported cstate change: term changed from 0 to 1, leader changed from <none> to e9e3de1593ae48d9b4223fc56c69d5aa (127.8.16.67). New cstate: current_term: 1 leader_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } health_report { overall_health: UNKNOWN } } }
W20260807 05:11:53.641676 23378 tablet.cc:2369] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:11:53.645882 23685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:53.639867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60144 (local address 127.8.16.67:46527)
0807 05:11:53.640069 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:53.640085 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:53.640101 (+ 16us) server_negotiation.cc:407] Connection header received
0807 05:11:53.640296 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:53.640299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:53.640400 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:53.640555 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:11:53.640933 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.642086 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.643692 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:53.644083 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:53.644225 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:53.644548 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:11:53.645280 (+ 732us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:11:53.645425 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:53.645517 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 05:11:53.645621 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 05:11:53.655954 23478 raft_consensus.cc:1261] T 081fceaa0d954a58a3ffcf1c31b953c6 P 93ccbf8081324583b623143d78125e41 [term 1 FOLLOWER]: Refusing update from remote peer e9e3de1593ae48d9b4223fc56c69d5aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:11:53.656082 23328 raft_consensus.cc:1261] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Refusing update from remote peer e9e3de1593ae48d9b4223fc56c69d5aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:11:53.657112 23693 consensus_queue.cc:1035] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:11:53.657398 23692 consensus_queue.cc:1035] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:11:53.669327 23698 mvcc.cc:205] Tried to move back new op lower bound from 7315781687921999872 to 7315781687748947968. Current Snapshot: MvccSnapshot[applied={T|T < 7315781687921999872}]
I20260807 05:11:53.686216 23175 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:45772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:11:53.686512 23175 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:45772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:11:53.690191 23175 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e: Sending DeleteTablet for 3 replicas of tablet 081fceaa0d954a58a3ffcf1c31b953c6
I20260807 05:11:53.691355 23601 tablet_service.cc:1514] Processing DeleteTablet for tablet 081fceaa0d954a58a3ffcf1c31b953c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:11:53 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:60134
I20260807 05:11:53.691725 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 23216
I20260807 05:11:53.691870 23456 tablet_service.cc:1514] Processing DeleteTablet for tablet 081fceaa0d954a58a3ffcf1c31b953c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:11:53 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:48770
I20260807 05:11:53.693516 23532 tablet_replica.cc:330] T 081fceaa0d954a58a3ffcf1c31b953c6 P 93ccbf8081324583b623143d78125e41: stopping tablet replica
I20260807 05:11:53.693746 23532 raft_consensus.cc:2229] T 081fceaa0d954a58a3ffcf1c31b953c6 P 93ccbf8081324583b623143d78125e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:11:53.693981 23532 raft_consensus.cc:2258] T 081fceaa0d954a58a3ffcf1c31b953c6 P 93ccbf8081324583b623143d78125e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:11:53.696372 23709 tablet_replica.cc:330] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa: stopping tablet replica
I20260807 05:11:53.696760 23532 ts_tablet_manager.cc:1905] T 081fceaa0d954a58a3ffcf1c31b953c6 P 93ccbf8081324583b623143d78125e41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:11:53.697211 23709 raft_consensus.cc:2229] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:11:53.698140 23709 raft_consensus.cc:2258] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:11:53.701159 23709 ts_tablet_manager.cc:1905] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:11:53.701573 23532 ts_tablet_manager.cc:1918] T 081fceaa0d954a58a3ffcf1c31b953c6 P 93ccbf8081324583b623143d78125e41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:11:53.701743 23532 log.cc:1196] T 081fceaa0d954a58a3ffcf1c31b953c6 P 93ccbf8081324583b623143d78125e41: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/wal/wals/081fceaa0d954a58a3ffcf1c31b953c6
I20260807 05:11:53.702301 23532 ts_tablet_manager.cc:1939] T 081fceaa0d954a58a3ffcf1c31b953c6 P 93ccbf8081324583b623143d78125e41: Deleting consensus metadata
I20260807 05:11:53.703684 23161 catalog_manager.cc:4841] TS 93ccbf8081324583b623143d78125e41 (127.8.16.66:46281): tablet 081fceaa0d954a58a3ffcf1c31b953c6 (table test-table [id=e22c1a8b310d43588b56b048cf6bb5e7]) successfully deleted
W20260807 05:11:53.704754 23161 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.16.65:32949 (error 108)
I20260807 05:11:53.705287 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 23382
I20260807 05:11:53.708103 23385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:53.707533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:32949 (local address 127.0.0.1:59604)
0807 05:11:53.707839 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:53.707952 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.65:32949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260807 05:11:53.708433 23709 ts_tablet_manager.cc:1918] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260807 05:11:53.708386 23161 catalog_manager.cc:4584] TS 88d6b5da8aba4197a622da1cbc66023d (127.8.16.65:32949): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 081fceaa0d954a58a3ffcf1c31b953c6: Network error: Client connection negotiation failed: client connection to 127.8.16.65:32949: connect: Connection refused (error 111)
I20260807 05:11:53.708698 23709 log.cc:1196] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/081fceaa0d954a58a3ffcf1c31b953c6
I20260807 05:11:53.709363 23709 ts_tablet_manager.cc:1939] T 081fceaa0d954a58a3ffcf1c31b953c6 P e9e3de1593ae48d9b4223fc56c69d5aa: Deleting consensus metadata
I20260807 05:11:53.711457 23159 catalog_manager.cc:4841] TS e9e3de1593ae48d9b4223fc56c69d5aa (127.8.16.67:46527): tablet 081fceaa0d954a58a3ffcf1c31b953c6 (table test-table [id=e22c1a8b310d43588b56b048cf6bb5e7]) successfully deleted
I20260807 05:11:53.715392 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 23529
I20260807 05:11:53.724575 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 23140
I20260807 05:11:53.735230 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:44131
--webserver_interface=127.8.16.126
--webserver_port=34171
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:35985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:44131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:11:53.928170 23711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:53.928678 23711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:53.928764 23711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:53.935240 23711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:11:53.935374 23711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:53.935442 23711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:53.935503 23711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:11:53.935556 23711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:11:53.947681 23711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:35985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:44131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:44131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=34171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:53.949690 23711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:53.951444 23711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:11:53.964696 23717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:53.964713 23718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:53.964696 23722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:53.965523 23711 server_base.cc:1034] running on GCE node
I20260807 05:11:53.966502 23711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:53.968454 23711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:53.969740 23711 hybrid_clock.cc:648] HybridClock initialized: now 1786079513969694 us; error 68 us; skew 500 ppm
Aug 07 05:11:53 dist-test-slave-49h3 krb5kdc[22548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079513, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:53.974364 23711 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:11:53.975950 23711 webserver.cc:460] Webserver started at http://127.8.16.126:34171/ using document root <none> and password file <none>
I20260807 05:11:53.976810 23711 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:53.976924 23711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:11:53.982908 23711 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260807 05:11:53.987720 23730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:53.991094 23711 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.006s sys 0.000s
I20260807 05:11:53.991328 23711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "6c5c1de2736e4027a7a6b15d06e19a1e"
format_stamp: "Formatted at 2026-08-07 05:11:51 on dist-test-slave-49h3"
server_key: "442cc7ac7135d34c381312558e5c6841"
server_key_iv: "7d0873c9f1275e5b520cce454bc753b2"
server_key_version: "encryptionkey@0"
I20260807 05:11:53.992709 23711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:54.037276 23711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:54.042273 23711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:54.042585 23711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:54.054776 23711 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:44131
I20260807 05:11:54.055255 23788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:44131 every 8 connection(s)
I20260807 05:11:54.056679 23711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:11:54.063707 23789 sys_catalog.cc:263] Verifying existing consensus state
I20260807 05:11:54.066519 23789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e: Bootstrap starting.
I20260807 05:11:54.066881 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 23711
I20260807 05:11:54.067682 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:32949
--local_ip_for_outbound_sockets=127.8.16.65
--tserver_master_addrs=127.8.16.126:44131
--webserver_port=42171
--webserver_interface=127.8.16.65
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:35985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:11:54.098510 23789 log.cc:824] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e: Log is configured to *not* fsync() on all Append() calls
I20260807 05:11:54.119190 23789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:11:54.119791 23789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e: Bootstrap complete.
I20260807 05:11:54.125911 23789 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44131 } }
I20260807 05:11:54.126662 23789 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c5c1de2736e4027a7a6b15d06e19a1e, State: Initialized, Role: FOLLOWER
I20260807 05:11:54.127492 23789 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44131 } }
I20260807 05:11:54.127667 23789 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:11:54.127795 23789 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:11:54.127960 23789 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 2 FOLLOWER]: Advancing to term 3
I20260807 05:11:54.131381 23789 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44131 } }
I20260807 05:11:54.131847 23789 leader_election.cc:302] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c5c1de2736e4027a7a6b15d06e19a1e; no voters:
I20260807 05:11:54.132401 23789 leader_election.cc:288] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 05:11:54.132498 23794 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 3 FOLLOWER]: Leader election won for term 3
I20260807 05:11:54.132833 23794 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [term 3 LEADER]: Becoming Leader. State: Replica: 6c5c1de2736e4027a7a6b15d06e19a1e, State: Running, Role: LEADER
I20260807 05:11:54.133436 23794 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44131 } }
I20260807 05:11:54.134650 23789 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:11:54.135182 23794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c5c1de2736e4027a7a6b15d06e19a1e. Latest consensus state: current_term: 3 leader_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44131 } } }
I20260807 05:11:54.135407 23794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [sys.catalog]: This master's current role is: LEADER
I20260807 05:11:54.136590 23795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5c1de2736e4027a7a6b15d06e19a1e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44131 } } }
I20260807 05:11:54.137017 23795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c5c1de2736e4027a7a6b15d06e19a1e [sys.catalog]: This master's current role is: LEADER
I20260807 05:11:54.137746 23798 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:11:54.147193 23798 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:11:54.148452 23798 catalog_manager.cc:1259] Loaded cluster ID: e9b118e990054a0aa4e104debe822d61
I20260807 05:11:54.148562 23798 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 05:11:54.150800 23798 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f5ad35de9ad google::LogMessage::Fail() at ??:0
@ 0x7f5ad35e188a google::LogMessage::SendToLog() at ??:0
@ 0x7f5ad35de4de google::LogMessage::Flush() at ??:0
@ 0x7f5ad35dfa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5addf1dfdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f5addf1ed2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f5addf1b579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f5addf5f640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f5ad58303d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f5ad5830c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f5ad58323f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f5ad5821512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5ad6e656db start_thread at ??:0
@ 0x7f5ad20dc61f clone at ??:0
W20260807 05:11:54.291339 23792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:54.291754 23792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:54.291867 23792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:54.298004 23792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:54.298156 23792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:54.298301 23792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:11:54.306428 23792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:35985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:32949
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=42171
--tserver_master_addrs=127.8.16.126:44131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:54.308180 23792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:54.309906 23792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:11:54.324653 23838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:54.324808 23836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:54.325059 23835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:54.325122 23792 server_base.cc:1034] running on GCE node
I20260807 05:11:54.326016 23792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:54.327337 23792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:54.328672 23792 hybrid_clock.cc:648] HybridClock initialized: now 1786079514328622 us; error 57 us; skew 500 ppm
Aug 07 05:11:54 dist-test-slave-49h3 krb5kdc[22548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079514, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:54.334848 23792 init.cc:377] Logged in from keytab as kudu/127.8.16.65@KRBTEST.COM (short username kudu)
I20260807 05:11:54.336282 23792 webserver.cc:460] Webserver started at http://127.8.16.65:42171/ using document root <none> and password file <none>
I20260807 05:11:54.337146 23792 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:54.337263 23792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:11:54.343539 23792 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 05:11:54.347049 23845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:54.349288 23792 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 05:11:54.349499 23792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "88d6b5da8aba4197a622da1cbc66023d"
format_stamp: "Formatted at 2026-08-07 05:11:51 on dist-test-slave-49h3"
server_key: "12c85cb5b76908b1f9af9f48d53c4897"
server_key_iv: "bf9fc055a5ab96ec809a178d9416b7b2"
server_key_version: "encryptionkey@0"
I20260807 05:11:54.350342 23792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:54.387929 23792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:54.394098 23792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:54.394485 23792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:54.395879 23792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:11:54.398293 23854 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 05:11:54.404831 23792 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 05:11:54.404968 23792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260807 05:11:54.405117 23792 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 05:11:54.409569 23792 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 05:11:54.409691 23792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.005s user 0.005s sys 0.000s
I20260807 05:11:54.410112 23854 tablet_bootstrap.cc:492] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d: Bootstrap starting.
I20260807 05:11:54.430979 23792 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:32949
I20260807 05:11:54.431449 23970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:32949 every 8 connection(s)
I20260807 05:11:54.433585 23792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:11:54.440750 23854 log.cc:824] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d: Log is configured to *not* fsync() on all Append() calls
I20260807 05:11:54.443413 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 23792
I20260807 05:11:54.444228 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:46281
--local_ip_for_outbound_sockets=127.8.16.66
--tserver_master_addrs=127.8.16.126:44131
--webserver_port=46183
--webserver_interface=127.8.16.66
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:35985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:11:54.452776 23854 tablet_bootstrap.cc:492] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:11:54.453315 23854 tablet_bootstrap.cc:492] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d: Bootstrap complete.
I20260807 05:11:54.454667 23854 ts_tablet_manager.cc:1397] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d: Time spent bootstrapping tablet: real 0.045s user 0.023s sys 0.020s
I20260807 05:11:54.458444 23854 raft_consensus.cc:348] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:54.459115 23854 raft_consensus.cc:729] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88d6b5da8aba4197a622da1cbc66023d, State: Initialized, Role: FOLLOWER
I20260807 05:11:54.459802 23854 consensus_queue.cc:260] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:54.461105 23854 ts_tablet_manager.cc:1428] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260807 05:11:54.523571 23973 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:11:54.437853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.65:43549)
0807 05:11:54.440152 (+ 2299us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:54.440295 (+ 143us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:54.442287 (+ 1992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:54.521757 (+ 79470us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1826,"thread_start_us":297,"threads_started":1}
W20260807 05:11:54.524631 23971 heartbeater.cc:644] Failed to heartbeat to 127.8.16.126:44131 (0 consecutive failures): Network error: Failed to ping master at 127.8.16.126:44131: Client connection negotiation failed: client connection to 127.8.16.126:44131: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 05:11:54.525856 23973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:54.525363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.65:47289)
0807 05:11:54.525608 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:54.525731 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 05:11:54.526588 23973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:54.526382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.65:51695)
0807 05:11:54.526480 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:54.526531 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 05:11:54.526749 23971 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 05:11:54.600270 23975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:54.600519 23975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:54.600561 23975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:54.604489 23975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:54.604552 23975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:54.604607 23975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:11:54.609815 23975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:35985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:46281
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=46183
--tserver_master_addrs=127.8.16.126:44131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:54.610816 23975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:54.611774 23975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:11:54.620844 23983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:54.620844 23982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:54.620972 23975 server_base.cc:1034] running on GCE node
W20260807 05:11:54.621188 23985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:54.621605 23975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:54.622179 23975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:54.623325 23975 hybrid_clock.cc:648] HybridClock initialized: now 1786079514623314 us; error 24 us; skew 500 ppm
Aug 07 05:11:54 dist-test-slave-49h3 krb5kdc[22548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079514, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:54.625794 23975 init.cc:377] Logged in from keytab as kudu/127.8.16.66@KRBTEST.COM (short username kudu)
I20260807 05:11:54.626631 23975 webserver.cc:460] Webserver started at http://127.8.16.66:46183/ using document root <none> and password file <none>
I20260807 05:11:54.627153 23975 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:54.627218 23975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:11:54.630967 23975 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 05:11:54.632961 23993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:54.634135 23975 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:11:54.634246 23975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "93ccbf8081324583b623143d78125e41"
format_stamp: "Formatted at 2026-08-07 05:11:51 on dist-test-slave-49h3"
server_key: "12b862660c003cdbc8804c2742ca4025"
server_key_iv: "99a040659bf31996dcf05def6e548ecb"
server_key_version: "encryptionkey@0"
I20260807 05:11:54.634757 23975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:54.650939 23975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:54.653589 23975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:54.653767 23975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:54.654349 23975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:11:54.655160 23975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:11:54.655224 23975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:54.655277 23975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:11:54.655303 23975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:54.663652 23975 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:46281
I20260807 05:11:54.663678 24117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:46281 every 8 connection(s)
I20260807 05:11:54.664605 23975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:11:54.666013 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 23975
I20260807 05:11:54.666428 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:46527
--local_ip_for_outbound_sockets=127.8.16.67
--tserver_master_addrs=127.8.16.126:44131
--webserver_port=39691
--webserver_interface=127.8.16.67
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:35985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:11:54.668504 24120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:54.666704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.66:54363)
0807 05:11:54.667620 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:54.667762 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":534,"threads_started":1}
W20260807 05:11:54.668912 24118 heartbeater.cc:644] Failed to heartbeat to 127.8.16.126:44131 (0 consecutive failures): Network error: Failed to ping master at 127.8.16.126:44131: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
I20260807 05:11:54.669523 24120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:54.669302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.66:39279)
0807 05:11:54.669399 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:54.669465 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 05:11:54.670073 24120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:54.669906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.66:58849)
0807 05:11:54.669989 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:54.670027 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 05:11:54.670217 24118 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 05:11:54.774670 24122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:11:54.774920 24122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:11:54.774962 24122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:11:54.778891 24122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:11:54.778945 24122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:11:54.779002 24122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:11:54.784045 24122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:35985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:46527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=39691
--tserver_master_addrs=127.8.16.126:44131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:11:54.784998 24122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:11:54.785981 24122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:11:54.794976 24128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:54.795094 24122 server_base.cc:1034] running on GCE node
W20260807 05:11:54.794972 24127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:11:54.794992 24130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:11:54.795642 24122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:11:54.796204 24122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:11:54.797353 24122 hybrid_clock.cc:648] HybridClock initialized: now 1786079514797340 us; error 31 us; skew 500 ppm
Aug 07 05:11:54 dist-test-slave-49h3 krb5kdc[22548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079514, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:11:54.799929 24122 init.cc:377] Logged in from keytab as kudu/127.8.16.67@KRBTEST.COM (short username kudu)
I20260807 05:11:54.800742 24122 webserver.cc:460] Webserver started at http://127.8.16.67:39691/ using document root <none> and password file <none>
I20260807 05:11:54.801252 24122 fs_manager.cc:359] Metadata directory not provided
I20260807 05:11:54.801321 24122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:11:54.804819 24122 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:11:54.806769 24138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:54.807883 24122 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:11:54.807982 24122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "e9e3de1593ae48d9b4223fc56c69d5aa"
format_stamp: "Formatted at 2026-08-07 05:11:52 on dist-test-slave-49h3"
server_key: "42dd5bb737a9c6f9196ce5b08ebfd628"
server_key_iv: "8c8d414f2c91c77e94a024034b3312cb"
server_key_version: "encryptionkey@0"
I20260807 05:11:54.808492 24122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:11:54.823226 24122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:11:54.825829 24122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:11:54.826000 24122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:11:54.826565 24122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:11:54.827349 24122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:11:54.827405 24122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:54.827457 24122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:11:54.827482 24122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:11:54.835954 24122 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:46527
I20260807 05:11:54.835975 24263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:46527 every 8 connection(s)
I20260807 05:11:54.836898 24122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:11:54.840566 24266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:54.839014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.67:34543)
0807 05:11:54.839713 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:54.839829 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":107,"threads_started":1}
W20260807 05:11:54.841092 24264 heartbeater.cc:644] Failed to heartbeat to 127.8.16.126:44131 (0 consecutive failures): Network error: Failed to ping master at 127.8.16.126:44131: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
I20260807 05:11:54.841702 24266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:54.841493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.67:51385)
0807 05:11:54.841597 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:54.841647 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 05:11:54.842291 24266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:54.842068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.67:33521)
0807 05:11:54.842190 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:54.842239 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 05:11:54.842446 24264 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 05:11:54.843046 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 24122
I20260807 05:11:55.527797 24269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:55.527293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.65:57039)
0807 05:11:55.527637 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:55.527692 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":74,"threads_started":1}
I20260807 05:11:55.671168 24270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:55.670764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.66:34807)
0807 05:11:55.671011 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:55.671068 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":75,"threads_started":1}
I20260807 05:11:55.843355 24271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:55.842962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.67:57511)
0807 05:11:55.843191 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:55.843248 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":68,"threads_started":1}
I20260807 05:11:56.078763 24272 raft_consensus.cc:482] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:11:56.078958 24272 raft_consensus.cc:504] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:56.080016 24272 leader_election.cc:288] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9e3de1593ae48d9b4223fc56c69d5aa (127.8.16.67:46527), 93ccbf8081324583b623143d78125e41 (127.8.16.66:46281)
Aug 07 05:11:56 dist-test-slave-49h3 krb5kdc[22548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079514, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.67@KRBTEST.COM
Aug 07 05:11:56 dist-test-slave-49h3 krb5kdc[22548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079514, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.66@KRBTEST.COM
I20260807 05:11:56.095947 24276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:56.080429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:57623 (local address 127.8.16.66:46281)
0807 05:11:56.080834 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:56.080859 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:56.080903 (+ 44us) server_negotiation.cc:407] Connection header received
0807 05:11:56.081815 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:56.081824 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:56.082267 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:56.082595 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:56.084246 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:56.085805 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:56.087613 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:56.088335 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:56.088493 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:56.091891 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:56.091897 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:56.091911 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:56.091945 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:56.093573 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:56.094351 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:56.094356 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:56.094359 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:56.094470 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:56.094723 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:56.094726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:56.094728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:56.095083 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:56.095190 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:56.095520 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 05:11:56.095776 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":87,"threads_started":1}
I20260807 05:11:56.095935 24273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:56.080246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:46281 (local address 127.8.16.65:57623)
0807 05:11:56.080576 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:56.080591 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:56.080708 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:56.082690 (+ 1982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:56.082696 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:56.083271 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:56.084044 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:56.084066 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:56.085933 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:56.085937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:56.087496 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:56.087508 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:56.088673 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:56.088676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:56.088941 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:56.089665 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:56.089693 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:56.091706 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:56.094053 (+ 2347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:56.094055 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:56.094056 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:56.094235 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:56.094566 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:56.094568 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:56.094569 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:56.094643 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:56.095193 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:56.095196 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:56.095282 (+ 86us) client_negotiation.cc:769] Sending connection context
0807 05:11:56.095392 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 05:11:56.095740 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":317,"spinlock_wait_cycles":29440,"thread_start_us":89,"threads_started":1}
I20260807 05:11:56.095947 24275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:56.080429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:40227 (local address 127.8.16.67:46527)
0807 05:11:56.080834 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0807 05:11:56.080859 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 05:11:56.080903 (+ 44us) server_negotiation.cc:407] Connection header received
0807 05:11:56.081815 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:11:56.081824 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:11:56.082267 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:11:56.082595 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:11:56.084246 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:56.085793 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:56.087613 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:11:56.088335 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:11:56.088493 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:56.091622 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:11:56.091626 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:11:56.091638 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:11:56.091677 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:11:56.093572 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:56.094052 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:56.094056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:56.094060 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:56.094178 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:11:56.094506 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:11:56.094510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:11:56.094512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:11:56.094902 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:11:56.095013 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 05:11:56.095515 (+ 502us) server_negotiation.cc:299] Negotiation successful
0807 05:11:56.095765 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":91,"threads_started":1}
I20260807 05:11:56.097294 24274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:56.080246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:46527 (local address 127.8.16.65:40227)
0807 05:11:56.080715 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:56.080727 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:11:56.080790 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:11:56.082690 (+ 1900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:11:56.082696 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:11:56.083282 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:11:56.084044 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:56.084066 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:56.085934 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:56.085937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:56.087498 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:11:56.087508 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:11:56.088672 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:11:56.088676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:11:56.088944 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:11:56.089665 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:11:56.089687 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:11:56.091401 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:11:56.093683 (+ 2282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:56.093688 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:56.093699 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:56.093961 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:56.094283 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:11:56.094286 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:11:56.094288 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:11:56.094420 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:11:56.095019 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:11:56.095026 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:11:56.095236 (+ 210us) client_negotiation.cc:769] Sending connection context
0807 05:11:56.095394 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 05:11:56.095814 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":491,"spinlock_wait_cycles":1792,"thread_start_us":81,"threads_started":1}
W20260807 05:11:56.098367 23849 leader_election.cc:341] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 93ccbf8081324583b623143d78125e41 (127.8.16.66:46281): Not found: Tablet not found: 081fceaa0d954a58a3ffcf1c31b953c6
W20260807 05:11:56.098901 23848 leader_election.cc:341] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e9e3de1593ae48d9b4223fc56c69d5aa (127.8.16.67:46527): Not found: Tablet not found: 081fceaa0d954a58a3ffcf1c31b953c6
I20260807 05:11:56.098989 23848 leader_election.cc:302] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88d6b5da8aba4197a622da1cbc66023d; no voters: 93ccbf8081324583b623143d78125e41, e9e3de1593ae48d9b4223fc56c69d5aa
I20260807 05:11:56.099255 24272 raft_consensus.cc:2726] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:11:56.528930 24274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:56.528636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.65:53649)
0807 05:11:56.528777 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:56.528841 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 05:11:56.672389 24277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:56.671905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.66:57191)
0807 05:11:56.672210 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:56.672277 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":79,"threads_started":1}
I20260807 05:11:56.844542 24278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:56.844048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.67:59579)
0807 05:11:56.844300 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:56.844431 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":94,"threads_started":1}
I20260807 05:11:57.530215 24279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:57.529736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.65:38695)
0807 05:11:57.530018 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:57.530112 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":79,"threads_started":1}
I20260807 05:11:57.673684 24280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:57.673179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.66:52573)
0807 05:11:57.673516 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:57.673582 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":80,"threads_started":1}
I20260807 05:11:57.845793 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:57.845334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.67:53991)
0807 05:11:57.845640 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:57.845698 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":100,"threads_started":1}
I20260807 05:11:57.946631 24284 raft_consensus.cc:482] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:11:57.946766 24284 raft_consensus.cc:504] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:11:57.947166 24284 leader_election.cc:288] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9e3de1593ae48d9b4223fc56c69d5aa (127.8.16.67:46527), 93ccbf8081324583b623143d78125e41 (127.8.16.66:46281)
W20260807 05:11:57.948028 23848 leader_election.cc:341] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e9e3de1593ae48d9b4223fc56c69d5aa (127.8.16.67:46527): Not found: Tablet not found: 081fceaa0d954a58a3ffcf1c31b953c6
W20260807 05:11:57.948197 23849 leader_election.cc:341] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 93ccbf8081324583b623143d78125e41 (127.8.16.66:46281): Not found: Tablet not found: 081fceaa0d954a58a3ffcf1c31b953c6
I20260807 05:11:57.948336 23849 leader_election.cc:302] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88d6b5da8aba4197a622da1cbc66023d; no voters: 93ccbf8081324583b623143d78125e41, e9e3de1593ae48d9b4223fc56c69d5aa
I20260807 05:11:57.948479 24284 raft_consensus.cc:2726] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:11:58.531574 24285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:58.531056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.65:55473)
0807 05:11:58.531369 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:58.531434 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":100,"threads_started":1}
I20260807 05:11:58.674788 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:58.674389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.66:41783)
0807 05:11:58.674639 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:58.674693 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":78,"threads_started":1}
I20260807 05:11:58.846980 24287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:58.846553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.67:35883)
0807 05:11:58.846821 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:58.846882 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":97,"threads_started":1}
I20260807 05:11:59.533039 24288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:59.532565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.65:55125)
0807 05:11:59.532879 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:59.532938 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":77,"threads_started":1}
I20260807 05:11:59.676036 24289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:59.675559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.66:58343)
0807 05:11:59.675877 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:59.675939 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":106,"threads_started":1}
I20260807 05:11:59.848189 24290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:11:59.847729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.67:41989)
0807 05:11:59.848030 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 05:11:59.848089 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":109,"threads_started":1}
I20260807 05:12:00.254619 24291 raft_consensus.cc:482] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:12:00.254763 24291 raft_consensus.cc:504] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:12:00.255249 24291 leader_election.cc:288] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9e3de1593ae48d9b4223fc56c69d5aa (127.8.16.67:46527), 93ccbf8081324583b623143d78125e41 (127.8.16.66:46281)
W20260807 05:12:00.256160 23848 leader_election.cc:341] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e9e3de1593ae48d9b4223fc56c69d5aa (127.8.16.67:46527): Not found: Tablet not found: 081fceaa0d954a58a3ffcf1c31b953c6
W20260807 05:12:00.256274 23849 leader_election.cc:341] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 93ccbf8081324583b623143d78125e41 (127.8.16.66:46281): Not found: Tablet not found: 081fceaa0d954a58a3ffcf1c31b953c6
I20260807 05:12:00.256381 23849 leader_election.cc:302] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88d6b5da8aba4197a622da1cbc66023d; no voters: 93ccbf8081324583b623143d78125e41, e9e3de1593ae48d9b4223fc56c69d5aa
I20260807 05:12:00.256546 24291 raft_consensus.cc:2726] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:12:00.534317 24292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:00.533847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.65:39987)
0807 05:12:00.534148 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:00.534213 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":86,"threads_started":1}
I20260807 05:12:00.677307 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:00.676843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.66:46645)
0807 05:12:00.677134 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:00.677194 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":84,"threads_started":1}
I20260807 05:12:00.849321 24294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:00.848879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.67:35687)
0807 05:12:00.849147 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:00.849217 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":77,"threads_started":1}
I20260807 05:12:01.535576 24295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:01.535108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.65:57557)
0807 05:12:01.535399 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:01.535463 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":82,"threads_started":1}
I20260807 05:12:01.678601 24297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:01.678098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.66:36067)
0807 05:12:01.678441 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:01.678502 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":77,"threads_started":1}
I20260807 05:12:01.850522 24298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:01.850026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.67:46751)
0807 05:12:01.850360 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:01.850424 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":77,"threads_started":1}
I20260807 05:12:02.536793 24299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:02.536328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.65:57409)
0807 05:12:02.536631 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:02.536693 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":77,"threads_started":1}
I20260807 05:12:02.679826 24300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:02.679369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.66:53605)
0807 05:12:02.679659 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:02.679726 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":79,"threads_started":1}
I20260807 05:12:02.851673 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:02.851270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.67:42319)
0807 05:12:02.851517 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:02.851576 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":74,"threads_started":1}
I20260807 05:12:03.409341 24302 raft_consensus.cc:482] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:12:03.409520 24302 raft_consensus.cc:504] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:12:03.409961 24302 leader_election.cc:288] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9e3de1593ae48d9b4223fc56c69d5aa (127.8.16.67:46527), 93ccbf8081324583b623143d78125e41 (127.8.16.66:46281)
W20260807 05:12:03.410742 23848 leader_election.cc:341] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e9e3de1593ae48d9b4223fc56c69d5aa (127.8.16.67:46527): Not found: Tablet not found: 081fceaa0d954a58a3ffcf1c31b953c6
W20260807 05:12:03.410856 23849 leader_election.cc:341] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 93ccbf8081324583b623143d78125e41 (127.8.16.66:46281): Not found: Tablet not found: 081fceaa0d954a58a3ffcf1c31b953c6
I20260807 05:12:03.410964 23849 leader_election.cc:302] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88d6b5da8aba4197a622da1cbc66023d; no voters: 93ccbf8081324583b623143d78125e41, e9e3de1593ae48d9b4223fc56c69d5aa
I20260807 05:12:03.411128 24302 raft_consensus.cc:2726] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:12:03.538036 24303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:03.537551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.65:45637)
0807 05:12:03.537821 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:03.537880 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":76,"threads_started":1}
I20260807 05:12:03.681032 24304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:03.680604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.66:59431)
0807 05:12:03.680875 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:03.680932 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":85,"threads_started":1}
I20260807 05:12:03.852939 24305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:03.852492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.67:55665)
0807 05:12:03.852780 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:03.852843 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":83,"threads_started":1}
I20260807 05:12:04.539207 24306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:04.538763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.65:40743)
0807 05:12:04.539019 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:04.539075 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":78,"threads_started":1}
I20260807 05:12:04.682324 24307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:04.681898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.66:59919)
0807 05:12:04.682160 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:04.682225 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":80,"threads_started":1}
I20260807 05:12:04.854351 24309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:04.853828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.67:54281)
0807 05:12:04.854150 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:04.854224 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":99,"threads_started":1}
I20260807 05:12:05.540323 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:05.539908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.65:44937)
0807 05:12:05.540165 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:05.540221 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":77,"threads_started":1}
I20260807 05:12:05.683714 24311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:05.683210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.66:39863)
0807 05:12:05.683529 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:05.683609 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":95,"threads_started":1}
I20260807 05:12:05.855654 24312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:05.855191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.67:59455)
0807 05:12:05.855493 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:05.855557 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":99,"threads_started":1}
I20260807 05:12:06.541657 24313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:06.541183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.65:59583)
0807 05:12:06.541493 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:06.541548 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":91,"threads_started":1}
I20260807 05:12:06.684918 24314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:06.684479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.66:33527)
0807 05:12:06.684760 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:06.684817 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":97,"threads_started":1}
I20260807 05:12:06.856976 24316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:06.856533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.67:45005)
0807 05:12:06.856815 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:06.856876 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":87,"threads_started":1}
I20260807 05:12:07.542908 24317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:07.542442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.65:36649)
0807 05:12:07.542740 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:07.542801 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":93,"threads_started":1}
I20260807 05:12:07.686228 24318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:07.685766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.66:54659)
0807 05:12:07.686048 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:07.686127 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":91,"threads_started":1}
I20260807 05:12:07.858182 24319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:07.857748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.67:41337)
0807 05:12:07.858021 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:07.858083 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":79,"threads_started":1}
I20260807 05:12:08.544160 24320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:08.543713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.65:56223)
0807 05:12:08.543981 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:08.544041 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":80,"threads_started":1}
I20260807 05:12:08.630157 24321 raft_consensus.cc:482] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:12:08.630300 24321 raft_consensus.cc:504] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e3de1593ae48d9b4223fc56c69d5aa" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 46527 } } peers { permanent_uuid: "93ccbf8081324583b623143d78125e41" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46281 } } peers { permanent_uuid: "88d6b5da8aba4197a622da1cbc66023d" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 32949 } }
I20260807 05:12:08.630715 24321 leader_election.cc:288] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9e3de1593ae48d9b4223fc56c69d5aa (127.8.16.67:46527), 93ccbf8081324583b623143d78125e41 (127.8.16.66:46281)
W20260807 05:12:08.631532 23849 leader_election.cc:341] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 93ccbf8081324583b623143d78125e41 (127.8.16.66:46281): Not found: Tablet not found: 081fceaa0d954a58a3ffcf1c31b953c6
W20260807 05:12:08.631682 23848 leader_election.cc:341] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e9e3de1593ae48d9b4223fc56c69d5aa (127.8.16.67:46527): Not found: Tablet not found: 081fceaa0d954a58a3ffcf1c31b953c6
I20260807 05:12:08.631744 23848 leader_election.cc:302] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88d6b5da8aba4197a622da1cbc66023d; no voters: 93ccbf8081324583b623143d78125e41, e9e3de1593ae48d9b4223fc56c69d5aa
I20260807 05:12:08.631892 24321 raft_consensus.cc:2726] T 081fceaa0d954a58a3ffcf1c31b953c6 P 88d6b5da8aba4197a622da1cbc66023d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:12:08.687409 24322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:08.686979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.66:45249)
0807 05:12:08.687256 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:08.687309 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":98,"threads_started":1}
I20260807 05:12:08.859479 24323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:08.858992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.67:36793)
0807 05:12:08.859315 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:08.859380 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":110,"threads_started":1}
I20260807 05:12:09.545619 24324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:09.545164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.65:54781)
0807 05:12:09.545431 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:09.545520 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":75,"threads_started":1}
I20260807 05:12:09.688864 24325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:09.688400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:44131 (local address 127.8.16.66:54917)
0807 05:12:09.688697 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:09.688762 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:44131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":77,"threads_started":1}
I20260807 05:12:09.844444 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 23792
I20260807 05:12:09.850514 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 23975
I20260807 05:12:09.856429 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 24122
2026-08-07T05:12:09Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19746 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:12:09 dist-test-slave-49h3 krb5kdc[24328](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:12:09 dist-test-slave-49h3 krb5kdc[24328](info): set up 2 sockets
Aug 07 05:12:09 dist-test-slave-49h3 krb5kdc[24328](info): commencing operation
krb5kdc: starting...
W20260807 05:12:10.850538 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.972s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:12:10 dist-test-slave-49h3 krb5kdc[24328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:12:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:12:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:12:10.970635 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:41973
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:41973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:12:11.080153 24343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:12:11.080427 24343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:12:11.080484 24343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:12:11.084475 24343 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 05:12:11.084528 24343 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 05:12:11.084549 24343 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 05:12:11.084568 24343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:12:11.084590 24343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:12:11.084620 24343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:12:11.084649 24343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:12:11.084673 24343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:12:11.089965 24343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:41973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:41973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:12:11.090961 24343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:12:11.091945 24343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:12:11.100224 24351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:12:11.100211 24343 server_base.cc:1034] running on GCE node
W20260807 05:12:11.100174 24349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:12:11.100248 24348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:12:11.100958 24343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:12:11.102043 24343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:12:11.103197 24343 hybrid_clock.cc:648] HybridClock initialized: now 1786079531103173 us; error 37 us; skew 500 ppm
Aug 07 05:12:11 dist-test-slave-49h3 krb5kdc[24328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079531, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:12:11.105822 24343 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:12:11.106690 24343 webserver.cc:460] Webserver started at http://127.8.16.126:33345/ using document root <none> and password file <none>
I20260807 05:12:11.107196 24343 fs_manager.cc:359] Metadata directory not provided
I20260807 05:12:11.107254 24343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:12:11.107460 24343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:12:11.109434 24343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "d412a46cca7143a7a1aad6a57dc4ba18"
format_stamp: "Formatted at 2026-08-07 05:12:11 on dist-test-slave-49h3"
server_key: "0dec8bb2e91febe04c8884d03cdf8989"
server_key_iv: "6795e39dce447ddf59fdbb7d66e260fa"
server_key_version: "encryptionkey@0"
I20260807 05:12:11.109997 24343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "d412a46cca7143a7a1aad6a57dc4ba18"
format_stamp: "Formatted at 2026-08-07 05:12:11 on dist-test-slave-49h3"
server_key: "0dec8bb2e91febe04c8884d03cdf8989"
server_key_iv: "6795e39dce447ddf59fdbb7d66e260fa"
server_key_version: "encryptionkey@0"
I20260807 05:12:11.113339 24343 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20260807 05:12:11.115542 24358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:11.116724 24343 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:12:11.116833 24343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "d412a46cca7143a7a1aad6a57dc4ba18"
format_stamp: "Formatted at 2026-08-07 05:12:11 on dist-test-slave-49h3"
server_key: "0dec8bb2e91febe04c8884d03cdf8989"
server_key_iv: "6795e39dce447ddf59fdbb7d66e260fa"
server_key_version: "encryptionkey@0"
I20260807 05:12:11.116925 24343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:12:11.143716 24343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:12:11.146795 24343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:12:11.146979 24343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:12:11.153513 24343 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:41973
I20260807 05:12:11.153784 24415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:41973 every 8 connection(s)
I20260807 05:12:11.154465 24343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:12:11.157068 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 24343
I20260807 05:12:11.157141 24416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:12:11.157179 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:12:11.157438 8257 external_mini_cluster.cc:1428] Setting key 27c6a198c335c1ca66a2aefa16f5a3a3
I20260807 05:12:11.164022 24416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18: Bootstrap starting.
Aug 07 05:12:11 dist-test-slave-49h3 krb5kdc[24328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:12:11.166636 24416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18: Neither blocks nor log segments found. Creating new log.
I20260807 05:12:11.167451 24416 log.cc:824] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18: Log is configured to *not* fsync() on all Append() calls
I20260807 05:12:11.169397 24416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18: No bootstrap required, opened a new log
I20260807 05:12:11.171509 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:11.158868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54096 (local address 127.8.16.126:41973)
0807 05:12:11.159371 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:11.159386 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:11.159423 (+ 37us) server_negotiation.cc:407] Connection header received
0807 05:12:11.160043 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:11.160059 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:11.160380 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:11.160660 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:12:11.161580 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.162619 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.163767 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.164274 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.164392 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:11.167097 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:12:11.167103 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:12:11.167118 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:12:11.167150 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:12:11.168777 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:11.169231 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:11.169235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:11.169239 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:11.169344 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:11.169656 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:11.169659 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:11.169660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:11.170034 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:12:11.170217 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:11.170498 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 05:12:11.170782 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":162,"threads_started":1}
I20260807 05:12:11.172986 24416 raft_consensus.cc:348] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41973 } }
I20260807 05:12:11.173197 24416 raft_consensus.cc:374] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:12:11.173267 24416 raft_consensus.cc:729] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d412a46cca7143a7a1aad6a57dc4ba18, State: Initialized, Role: FOLLOWER
I20260807 05:12:11.173672 24416 consensus_queue.cc:260] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41973 } }
I20260807 05:12:11.173802 24416 raft_consensus.cc:388] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:12:11.173859 24416 raft_consensus.cc:482] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:12:11.173947 24416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:12:11.174775 24416 raft_consensus.cc:504] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41973 } }
I20260807 05:12:11.175100 24416 leader_election.cc:302] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d412a46cca7143a7a1aad6a57dc4ba18; no voters:
I20260807 05:12:11.175412 24416 leader_election.cc:288] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:12:11.175532 24423 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:12:11.175746 24423 raft_consensus.cc:686] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 1 LEADER]: Becoming Leader. State: Replica: d412a46cca7143a7a1aad6a57dc4ba18, State: Running, Role: LEADER
I20260807 05:12:11.176055 24423 consensus_queue.cc:237] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41973 } }
I20260807 05:12:11.176493 24416 sys_catalog.cc:564] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:12:11.177613 24424 sys_catalog.cc:455] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41973 } } }
I20260807 05:12:11.177740 24424 sys_catalog.cc:458] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [sys.catalog]: This master's current role is: LEADER
I20260807 05:12:11.178012 24425 sys_catalog.cc:455] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d412a46cca7143a7a1aad6a57dc4ba18. Latest consensus state: current_term: 1 leader_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41973 } } }
I20260807 05:12:11.178112 24425 sys_catalog.cc:458] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [sys.catalog]: This master's current role is: LEADER
W20260807 05:12:11.181145 24441 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:12:11.181216 24441 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:12:11.181311 24434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:12:11.182241 24434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:12:11.188040 24434 catalog_manager.cc:1347] Generated new cluster ID: 3985e8fca84d40a7a434a40437e08ee3
I20260807 05:12:11.188115 24434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:12:11.196904 24434 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:12:11.197940 24434 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:12:11.204147 24434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18: Generated new TSK 0
I20260807 05:12:11.204979 24434 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
I20260807 05:12:11.241887 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:41973
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:12:11.351699 24448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:12:11.351977 24448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:12:11.352020 24448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:12:11.356094 24448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:12:11.356175 24448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:12:11.356237 24448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:12:11.361792 24448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:41973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:12:11.362932 24448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:12:11.363986 24448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:12:11.373556 24454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:12:11.373585 24456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:12:11.373606 24453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:12:11.373895 24448 server_base.cc:1034] running on GCE node
I20260807 05:12:11.374336 24448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:12:11.374987 24448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:12:11.376150 24448 hybrid_clock.cc:648] HybridClock initialized: now 1786079531376134 us; error 31 us; skew 500 ppm
Aug 07 05:12:11 dist-test-slave-49h3 krb5kdc[24328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079531, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:12:11.379074 24448 init.cc:377] Logged in from keytab as kudu/127.8.16.65@KRBTEST.COM (short username kudu)
I20260807 05:12:11.379953 24448 webserver.cc:460] Webserver started at http://127.8.16.65:33597/ using document root <none> and password file <none>
I20260807 05:12:11.380489 24448 fs_manager.cc:359] Metadata directory not provided
I20260807 05:12:11.380556 24448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:12:11.380751 24448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:12:11.382776 24448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "15a96e9746cf467d93b56735f7572af5"
format_stamp: "Formatted at 2026-08-07 05:12:11 on dist-test-slave-49h3"
server_key: "eb753fe2a4295f32ddaee063cf31770b"
server_key_iv: "91435cd66ef56cb135cf478cb19d5ea2"
server_key_version: "encryptionkey@0"
I20260807 05:12:11.383320 24448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "15a96e9746cf467d93b56735f7572af5"
format_stamp: "Formatted at 2026-08-07 05:12:11 on dist-test-slave-49h3"
server_key: "eb753fe2a4295f32ddaee063cf31770b"
server_key_iv: "91435cd66ef56cb135cf478cb19d5ea2"
server_key_version: "encryptionkey@0"
I20260807 05:12:11.386924 24448 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:12:11.389297 24465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:11.390515 24448 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:12:11.390623 24448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "15a96e9746cf467d93b56735f7572af5"
format_stamp: "Formatted at 2026-08-07 05:12:11 on dist-test-slave-49h3"
server_key: "eb753fe2a4295f32ddaee063cf31770b"
server_key_iv: "91435cd66ef56cb135cf478cb19d5ea2"
server_key_version: "encryptionkey@0"
I20260807 05:12:11.390725 24448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:12:11.404299 24448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:12:11.407042 24448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:12:11.407212 24448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:12:11.407754 24448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:12:11.408519 24448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:12:11.408565 24448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:11.408614 24448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:12:11.408643 24448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:11.416872 24448 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:38749
I20260807 05:12:11.416905 24588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:38749 every 8 connection(s)
I20260807 05:12:11.417726 24448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:12:11.418293 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 24448
I20260807 05:12:11.418404 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:12:11.418699 8257 external_mini_cluster.cc:1428] Setting key c15f15c88e037518f784ca49e51b5d21
WARNING: no policy specified for kudu/127.8.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.66@KRBTEST.COM" created.
Aug 07 05:12:11 dist-test-slave-49h3 krb5kdc[24328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079531, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:12:11.434639 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:11.419500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:50707 (local address 127.8.16.126:41973)
0807 05:12:11.419649 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:11.419653 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:11.422047 (+ 2394us) server_negotiation.cc:407] Connection header received
0807 05:12:11.422916 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:11.422919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:11.422975 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:11.423047 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:12:11.424207 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.424745 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.425912 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.426166 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.426256 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:11.430954 (+ 4698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:12:11.430959 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:12:11.430962 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:12:11.430995 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:12:11.432425 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:11.432945 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:11.432948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:11.432949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:11.433017 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:11.433382 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:11.433385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:11.433386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:11.433609 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:12:11.433684 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:11.434364 (+ 680us) server_negotiation.cc:299] Negotiation successful
0807 05:12:11.434508 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 05:12:11.435693 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:11.419882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.65:50707)
0807 05:12:11.421900 (+ 2018us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:11.421941 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:12:11.422716 (+ 775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:12:11.423241 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:12:11.423247 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:12:11.423593 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:12:11.424048 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:11.424060 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.424865 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.424867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:11.425791 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:11.425800 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.426310 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.426314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:11.426473 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:11.427017 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:12:11.427033 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:12:11.430753 (+ 3720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:12:11.432548 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:12:11.432554 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:12:11.432571 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:12:11.432861 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:12:11.433093 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:12:11.433096 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:12:11.433098 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:12:11.433273 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:12:11.433708 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:12:11.433714 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:12:11.434046 (+ 332us) client_negotiation.cc:769] Sending connection context
0807 05:12:11.434261 (+ 215us) client_negotiation.cc:240] Negotiation successful
0807 05:12:11.434619 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1839,"thread_start_us":110,"threads_started":1}
I20260807 05:12:11.436920 24589 heartbeater.cc:347] Connected to a master server at 127.8.16.126:41973
I20260807 05:12:11.437148 24589 heartbeater.cc:464] Registering TS with master...
I20260807 05:12:11.437665 24589 heartbeater.cc:511] Master 127.8.16.126:41973 requested a full tablet report, sending...
I20260807 05:12:11.439051 24376 ts_manager.cc:194] Registered new tserver with Master: 15a96e9746cf467d93b56735f7572af5 (127.8.16.65:38749)
I20260807 05:12:11.440160 24376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:50707
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
I20260807 05:12:11.445796 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:41973
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:12:11.554569 24595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:12:11.554859 24595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:12:11.554916 24595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:12:11.558959 24595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:12:11.559032 24595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:12:11.559118 24595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:12:11.564385 24595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:41973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:12:11.565512 24595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:12:11.566550 24595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:12:11.576079 24600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:12:11.576081 24603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:12:11.576236 24595 server_base.cc:1034] running on GCE node
W20260807 05:12:11.576077 24601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:12:11.576777 24595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:12:11.577487 24595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:12:11.578649 24595 hybrid_clock.cc:648] HybridClock initialized: now 1786079531578631 us; error 35 us; skew 500 ppm
Aug 07 05:12:11 dist-test-slave-49h3 krb5kdc[24328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079531, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:12:11.581422 24595 init.cc:377] Logged in from keytab as kudu/127.8.16.66@KRBTEST.COM (short username kudu)
I20260807 05:12:11.582419 24595 webserver.cc:460] Webserver started at http://127.8.16.66:36861/ using document root <none> and password file <none>
I20260807 05:12:11.582948 24595 fs_manager.cc:359] Metadata directory not provided
I20260807 05:12:11.583007 24595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:12:11.583174 24595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:12:11.585186 24595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "0d4b47e627f24cef9e22a5c6bf04bb05"
format_stamp: "Formatted at 2026-08-07 05:12:11 on dist-test-slave-49h3"
server_key: "c6f0933fe8a13eb007d0521140c1253b"
server_key_iv: "0d57ecc52845f1f05c8c2350331cc3ec"
server_key_version: "encryptionkey@0"
I20260807 05:12:11.585749 24595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "0d4b47e627f24cef9e22a5c6bf04bb05"
format_stamp: "Formatted at 2026-08-07 05:12:11 on dist-test-slave-49h3"
server_key: "c6f0933fe8a13eb007d0521140c1253b"
server_key_iv: "0d57ecc52845f1f05c8c2350331cc3ec"
server_key_version: "encryptionkey@0"
I20260807 05:12:11.589389 24595 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 05:12:11.591777 24611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:11.593102 24595 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 05:12:11.593227 24595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "0d4b47e627f24cef9e22a5c6bf04bb05"
format_stamp: "Formatted at 2026-08-07 05:12:11 on dist-test-slave-49h3"
server_key: "c6f0933fe8a13eb007d0521140c1253b"
server_key_iv: "0d57ecc52845f1f05c8c2350331cc3ec"
server_key_version: "encryptionkey@0"
I20260807 05:12:11.593346 24595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:12:11.608359 24595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:12:11.611379 24595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:12:11.611574 24595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:12:11.612218 24595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:12:11.613073 24595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:12:11.613123 24595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:11.613178 24595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:12:11.613207 24595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:11.622063 24595 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:46191
I20260807 05:12:11.622114 24732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:46191 every 8 connection(s)
I20260807 05:12:11.622956 24595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
Aug 07 05:12:11 dist-test-slave-49h3 krb5kdc[24328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079531, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:12:11.632520 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 24595
I20260807 05:12:11.632627 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:12:11.632895 8257 external_mini_cluster.cc:1428] Setting key ecdab915c28b149a2dfa783b6aeb0f11
I20260807 05:12:11.635584 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:11.624677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:48351 (local address 127.8.16.126:41973)
0807 05:12:11.624818 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:11.624822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:11.625888 (+ 1066us) server_negotiation.cc:407] Connection header received
0807 05:12:11.626683 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:11.626686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:11.626732 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:11.626803 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:12:11.627848 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.628381 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.629419 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.629689 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.629764 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:11.632133 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:12:11.632137 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:12:11.632139 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:12:11.632165 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:12:11.633669 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:11.634148 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:11.634152 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:11.634153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:11.634228 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:11.634543 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:11.634547 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:11.634548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:11.634727 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:12:11.634812 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:11.635293 (+ 481us) server_negotiation.cc:299] Negotiation successful
0807 05:12:11.635416 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 05:12:11.636548 24735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:11.625074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.66:48351)
0807 05:12:11.625741 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:11.625791 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 05:12:11.626522 (+ 731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:12:11.626949 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:12:11.626955 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:12:11.627289 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:12:11.627735 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:11.627747 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.628467 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.628470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:11.629332 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:11.629341 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.629811 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.629814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:11.629960 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:11.630511 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:12:11.630528 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:12:11.631914 (+ 1386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:12:11.633783 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:12:11.633789 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:12:11.633804 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:12:11.634047 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:12:11.634313 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:12:11.634316 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:12:11.634318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:12:11.634455 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:12:11.634831 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:12:11.634837 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:12:11.635062 (+ 225us) client_negotiation.cc:769] Sending connection context
0807 05:12:11.635229 (+ 167us) client_negotiation.cc:240] Negotiation successful
0807 05:12:11.635545 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":121,"threads_started":1}
I20260807 05:12:11.637718 24733 heartbeater.cc:347] Connected to a master server at 127.8.16.126:41973
I20260807 05:12:11.637944 24733 heartbeater.cc:464] Registering TS with master...
I20260807 05:12:11.638396 24733 heartbeater.cc:511] Master 127.8.16.126:41973 requested a full tablet report, sending...
I20260807 05:12:11.639451 24376 ts_manager.cc:194] Registered new tserver with Master: 0d4b47e627f24cef9e22a5c6bf04bb05 (127.8.16.66:46191)
I20260807 05:12:11.639990 24376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:48351
WARNING: no policy specified for kudu/127.8.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
I20260807 05:12:11.658583 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:41973
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:12:11.769317 24739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:12:11.769635 24739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:12:11.769706 24739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:12:11.773711 24739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:12:11.773777 24739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:12:11.773859 24739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:12:11.779242 24739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:41973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:12:11.780332 24739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:12:11.781368 24739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:12:11.790518 24745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:12:11.790556 24739 server_base.cc:1034] running on GCE node
W20260807 05:12:11.790506 24747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:12:11.790540 24744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:12:11.791177 24739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:12:11.791769 24739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:12:11.792925 24739 hybrid_clock.cc:648] HybridClock initialized: now 1786079531792901 us; error 39 us; skew 500 ppm
Aug 07 05:12:11 dist-test-slave-49h3 krb5kdc[24328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079531, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:12:11.795627 24739 init.cc:377] Logged in from keytab as kudu/127.8.16.67@KRBTEST.COM (short username kudu)
I20260807 05:12:11.796487 24739 webserver.cc:460] Webserver started at http://127.8.16.67:44625/ using document root <none> and password file <none>
I20260807 05:12:11.797021 24739 fs_manager.cc:359] Metadata directory not provided
I20260807 05:12:11.797092 24739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:12:11.797268 24739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:12:11.799341 24739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "18fd5bc0291745f7874ea50a73a243b3"
format_stamp: "Formatted at 2026-08-07 05:12:11 on dist-test-slave-49h3"
server_key: "80248c25c3a6cc7cd49e8699e9e11837"
server_key_iv: "710a7dbc29458e275700e26f5a3a88bb"
server_key_version: "encryptionkey@0"
I20260807 05:12:11.799849 24739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "18fd5bc0291745f7874ea50a73a243b3"
format_stamp: "Formatted at 2026-08-07 05:12:11 on dist-test-slave-49h3"
server_key: "80248c25c3a6cc7cd49e8699e9e11837"
server_key_iv: "710a7dbc29458e275700e26f5a3a88bb"
server_key_version: "encryptionkey@0"
I20260807 05:12:11.803478 24739 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:12:11.805760 24755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:11.806974 24739 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:12:11.807106 24739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "18fd5bc0291745f7874ea50a73a243b3"
format_stamp: "Formatted at 2026-08-07 05:12:11 on dist-test-slave-49h3"
server_key: "80248c25c3a6cc7cd49e8699e9e11837"
server_key_iv: "710a7dbc29458e275700e26f5a3a88bb"
server_key_version: "encryptionkey@0"
I20260807 05:12:11.807209 24739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:12:11.819994 24739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:12:11.822973 24739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:12:11.823149 24739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:12:11.823704 24739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:12:11.824673 24739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:12:11.824733 24739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:11.824785 24739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:12:11.824811 24739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:11.833539 24739 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:38607
I20260807 05:12:11.833567 24878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:38607 every 8 connection(s)
I20260807 05:12:11.834408 24739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
Aug 07 05:12:11 dist-test-slave-49h3 krb5kdc[24328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079531, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:12:11.844686 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 24739
I20260807 05:12:11.844828 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:12:11.845108 8257 external_mini_cluster.cc:1428] Setting key aa0ea60fe98ce656feb4acb3c3cb321d
I20260807 05:12:11.847579 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:11.836137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:44617 (local address 127.8.16.126:41973)
0807 05:12:11.836266 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:11.836270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:11.837281 (+ 1011us) server_negotiation.cc:407] Connection header received
0807 05:12:11.838183 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:11.838187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:11.838247 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:11.838323 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:12:11.839479 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.840014 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.841150 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.841371 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.841487 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:11.843876 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:12:11.843879 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:12:11.843881 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:12:11.843916 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:12:11.845340 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:11.845835 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:11.845837 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:11.845838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:11.845924 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:11.846344 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:11.846347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:11.846348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:11.846563 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:12:11.846667 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:11.847303 (+ 636us) server_negotiation.cc:299] Negotiation successful
0807 05:12:11.847445 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 05:12:11.848418 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:11.836503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.67:44617)
0807 05:12:11.837131 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:11.837171 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 05:12:11.837977 (+ 806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:12:11.838499 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:12:11.838506 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:12:11.838850 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:12:11.839327 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:11.839341 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.840140 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.840142 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:11.841023 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:11.841032 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.841545 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.841549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:11.841706 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:11.842274 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:12:11.842290 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:12:11.843691 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:12:11.845469 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:12:11.845475 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:12:11.845491 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:12:11.845746 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:12:11.846025 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:12:11.846028 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:12:11.846030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:12:11.846231 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:12:11.846686 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:12:11.846693 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:12:11.847033 (+ 340us) client_negotiation.cc:769] Sending connection context
0807 05:12:11.847238 (+ 205us) client_negotiation.cc:240] Negotiation successful
0807 05:12:11.847573 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":116,"threads_started":1}
I20260807 05:12:11.849434 24879 heartbeater.cc:347] Connected to a master server at 127.8.16.126:41973
I20260807 05:12:11.849689 24879 heartbeater.cc:464] Registering TS with master...
I20260807 05:12:11.850152 24879 heartbeater.cc:511] Master 127.8.16.126:41973 requested a full tablet report, sending...
I20260807 05:12:11.851070 24376 ts_manager.cc:194] Registered new tserver with Master: 18fd5bc0291745f7874ea50a73a243b3 (127.8.16.67:38607)
I20260807 05:12:11.851598 24376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:44617
I20260807 05:12:11.859095 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:12:11.868064 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:11.861061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54098 (local address 127.8.16.126:41973)
0807 05:12:11.861215 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:11.861220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:11.861386 (+ 166us) server_negotiation.cc:407] Connection header received
0807 05:12:11.861521 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:11.861524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:11.861565 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:11.861640 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:12:11.862356 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.862960 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.863832 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.864045 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.864138 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:11.865370 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:12:11.865374 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:12:11.865376 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:12:11.865398 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:12:11.866511 (+ 1113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:11.866903 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:11.866907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:11.866909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:11.866986 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:11.867234 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:11.867237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:11.867239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:11.867452 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:12:11.867569 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:11.867806 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 05:12:11.867915 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 05:12:11.870736 24376 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54098:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:12:11.873682 24376 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:12:11.887444 24892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:11.881696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56390 (local address 127.8.16.66:46191)
0807 05:12:11.882665 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:11.882677 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:11.882700 (+ 23us) server_negotiation.cc:407] Connection header received
0807 05:12:11.882776 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:11.882782 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:11.882989 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:11.883131 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:12:11.883702 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.884455 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.885955 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.886805 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.886893 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:11.887044 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:11.887206 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 05:12:11.887295 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":853,"thread_start_us":130,"threads_started":1}
I20260807 05:12:11.887552 24893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:11.881691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:38749 (local address 127.0.0.1:47022)
0807 05:12:11.882371 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:11.882384 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:12:11.882476 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:12:11.883326 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:12:11.883332 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:12:11.883365 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:12:11.883617 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:11.883625 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.885116 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.885120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:11.886240 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:11.886250 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.887125 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.887129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:11.887218 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:11.887243 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 05:12:11.887306 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 05:12:11.887378 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"mutex_wait_us":135,"spinlock_wait_cycles":5760,"thread_start_us":112,"threads_started":1}
I20260807 05:12:11.887434 24891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:11.881522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:46191 (local address 127.0.0.1:56390)
0807 05:12:11.882190 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:11.882242 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 05:12:11.882367 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:12:11.883326 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:12:11.883332 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:12:11.883368 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:12:11.883576 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:11.883584 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.884590 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.884594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:11.885821 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:11.885831 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.886911 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.886914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:11.887016 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:11.887051 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 05:12:11.887160 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 05:12:11.887285 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":54,"spinlock_wait_cycles":6016,"thread_start_us":82,"threads_started":1}
I20260807 05:12:11.887923 24895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:11.882336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47022 (local address 127.8.16.65:38749)
0807 05:12:11.882882 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:11.882894 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:11.882915 (+ 21us) server_negotiation.cc:407] Connection header received
0807 05:12:11.882980 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:11.882984 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:11.883155 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:11.883434 (+ 279us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:12:11.883752 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.884952 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.886355 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.886995 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.887510 (+ 515us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:11.887646 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:11.887695 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 05:12:11.887777 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":123,"threads_started":1}
I20260807 05:12:11.888100 24890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:11.881898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:38607 (local address 127.0.0.1:34242)
0807 05:12:11.882632 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:11.882643 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:12:11.882710 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:12:11.883476 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:12:11.883479 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:12:11.883495 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:12:11.883622 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:11.883627 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.885461 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.885465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:11.886580 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:11.886591 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.887728 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.887732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:11.887825 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:11.887847 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 05:12:11.887913 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 05:12:11.887977 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":29,"thread_start_us":95,"threads_started":1}
I20260807 05:12:11.888535 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:11.882298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34242 (local address 127.8.16.67:38607)
0807 05:12:11.882895 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:11.882906 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:11.882928 (+ 22us) server_negotiation.cc:407] Connection header received
0807 05:12:11.882999 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:11.883003 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:11.883187 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:11.883322 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:12:11.883747 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.884957 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.886712 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.887480 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.887565 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:11.887670 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:11.888310 (+ 640us) server_negotiation.cc:299] Negotiation successful
0807 05:12:11.888401 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":142,"threads_started":1}
I20260807 05:12:11.890847 24806 tablet_service.cc:1467] Processing CreateTablet for tablet dfb1250196c34eee969d845ec4edaec4 (DEFAULT_TABLE table=test-table [id=f7f24e8e32bd41d1b06fd8b17d56a14c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:12:11.890887 24662 tablet_service.cc:1467] Processing CreateTablet for tablet dfb1250196c34eee969d845ec4edaec4 (DEFAULT_TABLE table=test-table [id=f7f24e8e32bd41d1b06fd8b17d56a14c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:12:11.890887 24519 tablet_service.cc:1467] Processing CreateTablet for tablet dfb1250196c34eee969d845ec4edaec4 (DEFAULT_TABLE table=test-table [id=f7f24e8e32bd41d1b06fd8b17d56a14c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:12:11.891819 24806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfb1250196c34eee969d845ec4edaec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:12:11.891891 24662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfb1250196c34eee969d845ec4edaec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:12:11.891891 24519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfb1250196c34eee969d845ec4edaec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:12:11.898665 24896 tablet_bootstrap.cc:492] T dfb1250196c34eee969d845ec4edaec4 P 18fd5bc0291745f7874ea50a73a243b3: Bootstrap starting.
I20260807 05:12:11.899897 24898 tablet_bootstrap.cc:492] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05: Bootstrap starting.
I20260807 05:12:11.901264 24896 tablet_bootstrap.cc:654] T dfb1250196c34eee969d845ec4edaec4 P 18fd5bc0291745f7874ea50a73a243b3: Neither blocks nor log segments found. Creating new log.
I20260807 05:12:11.901787 24897 tablet_bootstrap.cc:492] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5: Bootstrap starting.
I20260807 05:12:11.901969 24898 tablet_bootstrap.cc:654] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05: Neither blocks nor log segments found. Creating new log.
I20260807 05:12:11.902053 24896 log.cc:824] T dfb1250196c34eee969d845ec4edaec4 P 18fd5bc0291745f7874ea50a73a243b3: Log is configured to *not* fsync() on all Append() calls
I20260807 05:12:11.902757 24898 log.cc:824] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05: Log is configured to *not* fsync() on all Append() calls
I20260807 05:12:11.903838 24897 tablet_bootstrap.cc:654] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5: Neither blocks nor log segments found. Creating new log.
I20260807 05:12:11.904115 24896 tablet_bootstrap.cc:492] T dfb1250196c34eee969d845ec4edaec4 P 18fd5bc0291745f7874ea50a73a243b3: No bootstrap required, opened a new log
I20260807 05:12:11.904305 24896 ts_tablet_manager.cc:1397] T dfb1250196c34eee969d845ec4edaec4 P 18fd5bc0291745f7874ea50a73a243b3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 05:12:11.905012 24898 tablet_bootstrap.cc:492] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05: No bootstrap required, opened a new log
I20260807 05:12:11.905031 24897 log.cc:824] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5: Log is configured to *not* fsync() on all Append() calls
I20260807 05:12:11.905200 24898 ts_tablet_manager.cc:1397] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 05:12:11.906985 24897 tablet_bootstrap.cc:492] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5: No bootstrap required, opened a new log
I20260807 05:12:11.907178 24897 ts_tablet_manager.cc:1397] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 05:12:11.907411 24896 raft_consensus.cc:348] T dfb1250196c34eee969d845ec4edaec4 P 18fd5bc0291745f7874ea50a73a243b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } }
I20260807 05:12:11.907644 24896 raft_consensus.cc:374] T dfb1250196c34eee969d845ec4edaec4 P 18fd5bc0291745f7874ea50a73a243b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:12:11.907716 24896 raft_consensus.cc:729] T dfb1250196c34eee969d845ec4edaec4 P 18fd5bc0291745f7874ea50a73a243b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18fd5bc0291745f7874ea50a73a243b3, State: Initialized, Role: FOLLOWER
I20260807 05:12:11.908133 24896 consensus_queue.cc:260] T dfb1250196c34eee969d845ec4edaec4 P 18fd5bc0291745f7874ea50a73a243b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } }
I20260807 05:12:11.908730 24879 heartbeater.cc:503] Master 127.8.16.126:41973 was elected leader, sending a full tablet report...
I20260807 05:12:11.908900 24898 raft_consensus.cc:348] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } }
I20260807 05:12:11.909160 24898 raft_consensus.cc:374] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:12:11.909264 24898 raft_consensus.cc:729] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d4b47e627f24cef9e22a5c6bf04bb05, State: Initialized, Role: FOLLOWER
I20260807 05:12:11.909723 24898 consensus_queue.cc:260] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } }
I20260807 05:12:11.910076 24896 ts_tablet_manager.cc:1428] T dfb1250196c34eee969d845ec4edaec4 P 18fd5bc0291745f7874ea50a73a243b3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 05:12:11.910691 24898 ts_tablet_manager.cc:1428] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 05:12:11.910790 24897 raft_consensus.cc:348] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } }
I20260807 05:12:11.911026 24897 raft_consensus.cc:374] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:12:11.911052 24733 heartbeater.cc:503] Master 127.8.16.126:41973 was elected leader, sending a full tablet report...
I20260807 05:12:11.911080 24897 raft_consensus.cc:729] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15a96e9746cf467d93b56735f7572af5, State: Initialized, Role: FOLLOWER
I20260807 05:12:11.911450 24897 consensus_queue.cc:260] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } }
I20260807 05:12:11.912269 24589 heartbeater.cc:503] Master 127.8.16.126:41973 was elected leader, sending a full tablet report...
I20260807 05:12:11.912350 24897 ts_tablet_manager.cc:1428] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260807 05:12:11.918735 24590 tablet.cc:2369] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:12:11.972956 24903 raft_consensus.cc:482] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:12:11.973196 24903 raft_consensus.cc:504] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } }
I20260807 05:12:11.974401 24903 leader_election.cc:288] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15a96e9746cf467d93b56735f7572af5 (127.8.16.65:38749), 18fd5bc0291745f7874ea50a73a243b3 (127.8.16.67:38607)
I20260807 05:12:11.978782 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:11.974798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:50733 (local address 127.8.16.67:38607)
0807 05:12:11.974946 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:11.974951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:11.974970 (+ 19us) server_negotiation.cc:407] Connection header received
0807 05:12:11.975030 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:11.975033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:11.975088 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:11.975170 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:12:11.975441 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.976208 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.977664 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.978204 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.978306 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:11.978376 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:11.978590 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 05:12:11.978667 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 05:12:11.978783 24735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:11.974635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:38607 (local address 127.8.16.66:50733)
0807 05:12:11.974778 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:11.974795 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:12:11.974902 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:12:11.975187 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:12:11.975190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:12:11.975209 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:12:11.975337 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:11.975343 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.976306 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.976308 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:11.977551 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:11.977562 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.978341 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.978344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:11.978453 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:11.978481 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 05:12:11.978562 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 05:12:11.978641 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":1}
I20260807 05:12:11.978987 24905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:11.974635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:38749 (local address 127.8.16.66:57639)
0807 05:12:11.975228 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:11.975240 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:12:11.975331 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:12:11.975557 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:12:11.975559 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:12:11.975580 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:12:11.975668 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:11.975672 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.976518 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.976521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:11.977771 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:11.977778 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.978608 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.978612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:11.978702 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:11.978725 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:12:11.978802 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 05:12:11.978863 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":261,"thread_start_us":120,"threads_started":1}
I20260807 05:12:11.979004 24895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:11.974827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:57639 (local address 127.8.16.65:38749)
0807 05:12:11.975068 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:11.975081 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:11.975319 (+ 238us) server_negotiation.cc:407] Connection header received
0807 05:12:11.975419 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:11.975422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:11.975472 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:11.975544 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:12:11.975757 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.976422 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.977877 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:11.978492 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:11.978589 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:11.978647 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:11.978827 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 05:12:11.978898 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
I20260807 05:12:11.979616 24542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb1250196c34eee969d845ec4edaec4" candidate_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15a96e9746cf467d93b56735f7572af5" is_pre_election: true
I20260807 05:12:11.979869 24827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb1250196c34eee969d845ec4edaec4" candidate_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18fd5bc0291745f7874ea50a73a243b3" is_pre_election: true
I20260807 05:12:11.979979 24542 raft_consensus.cc:2454] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d4b47e627f24cef9e22a5c6bf04bb05 in term 0.
I20260807 05:12:11.980163 24827 raft_consensus.cc:2454] T dfb1250196c34eee969d845ec4edaec4 P 18fd5bc0291745f7874ea50a73a243b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d4b47e627f24cef9e22a5c6bf04bb05 in term 0.
I20260807 05:12:11.980489 24615 leader_election.cc:302] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d4b47e627f24cef9e22a5c6bf04bb05, 15a96e9746cf467d93b56735f7572af5; no voters:
I20260807 05:12:11.980800 24903 raft_consensus.cc:2781] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:12:11.980883 24903 raft_consensus.cc:482] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:12:11.980933 24903 raft_consensus.cc:3037] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:12:11.982062 24903 raft_consensus.cc:504] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } }
I20260807 05:12:11.982398 24903 leader_election.cc:288] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05 [CANDIDATE]: Term 1 election: Requested vote from peers 15a96e9746cf467d93b56735f7572af5 (127.8.16.65:38749), 18fd5bc0291745f7874ea50a73a243b3 (127.8.16.67:38607)
I20260807 05:12:11.982774 24827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb1250196c34eee969d845ec4edaec4" candidate_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18fd5bc0291745f7874ea50a73a243b3"
I20260807 05:12:11.982906 24542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb1250196c34eee969d845ec4edaec4" candidate_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15a96e9746cf467d93b56735f7572af5"
I20260807 05:12:11.982901 24827 raft_consensus.cc:3037] T dfb1250196c34eee969d845ec4edaec4 P 18fd5bc0291745f7874ea50a73a243b3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:12:11.983033 24542 raft_consensus.cc:3037] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:12:11.983945 24827 raft_consensus.cc:2454] T dfb1250196c34eee969d845ec4edaec4 P 18fd5bc0291745f7874ea50a73a243b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d4b47e627f24cef9e22a5c6bf04bb05 in term 1.
I20260807 05:12:11.984007 24542 raft_consensus.cc:2454] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d4b47e627f24cef9e22a5c6bf04bb05 in term 1.
I20260807 05:12:11.984313 24615 leader_election.cc:302] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d4b47e627f24cef9e22a5c6bf04bb05, 15a96e9746cf467d93b56735f7572af5; no voters:
I20260807 05:12:11.984503 24903 raft_consensus.cc:2781] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:12:11.984747 24903 raft_consensus.cc:686] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05 [term 1 LEADER]: Becoming Leader. State: Replica: 0d4b47e627f24cef9e22a5c6bf04bb05, State: Running, Role: LEADER
I20260807 05:12:11.985077 24903 consensus_queue.cc:237] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } }
I20260807 05:12:11.988155 24374 catalog_manager.cc:5495] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d4b47e627f24cef9e22a5c6bf04bb05 (127.8.16.66). New cstate: current_term: 1 leader_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260807 05:12:12.011744 24892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:12.008553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56406 (local address 127.8.16.66:46191)
0807 05:12:12.008671 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:12.008674 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:12.008700 (+ 26us) server_negotiation.cc:407] Connection header received
0807 05:12:12.008766 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:12.008768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:12.008819 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:12.008895 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:12:12.009100 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:12.009642 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:12.010483 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:12.010684 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:12.010754 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:12.010976 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:12:12.011401 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:12:12.011482 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:12.011540 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 05:12:12.011588 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:12:12.017351 24542 raft_consensus.cc:1261] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5 [term 1 FOLLOWER]: Refusing update from remote peer 0d4b47e627f24cef9e22a5c6bf04bb05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:12:12.017349 24827 raft_consensus.cc:1261] T dfb1250196c34eee969d845ec4edaec4 P 18fd5bc0291745f7874ea50a73a243b3 [term 1 FOLLOWER]: Refusing update from remote peer 0d4b47e627f24cef9e22a5c6bf04bb05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:12:12.018158 24903 consensus_queue.cc:1035] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:12:12.018364 24906 consensus_queue.cc:1035] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:12:12.026006 24911 mvcc.cc:205] Tried to move back new op lower bound from 7315781763135823872 to 7315781763013693440. Current Snapshot: MvccSnapshot[applied={T|T < 7315781763135823872}]
I20260807 05:12:12.026850 24910 mvcc.cc:205] Tried to move back new op lower bound from 7315781763135823872 to 7315781763013693440. Current Snapshot: MvccSnapshot[applied={T|T < 7315781763135823872}]
I20260807 05:12:12.034796 24376 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:12:12.034962 24376 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:12:12.037400 24376 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18: Sending DeleteTablet for 3 replicas of tablet dfb1250196c34eee969d845ec4edaec4
I20260807 05:12:12.038244 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 24448
I20260807 05:12:12.038208 24519 tablet_service.cc:1514] Processing DeleteTablet for tablet dfb1250196c34eee969d845ec4edaec4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:12:12 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:47022
I20260807 05:12:12.038206 24662 tablet_service.cc:1514] Processing DeleteTablet for tablet dfb1250196c34eee969d845ec4edaec4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:12:12 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:56390
I20260807 05:12:12.039121 24806 tablet_service.cc:1514] Processing DeleteTablet for tablet dfb1250196c34eee969d845ec4edaec4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:12:12 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:34242
I20260807 05:12:12.040923 24918 tablet_replica.cc:330] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05: stopping tablet replica
I20260807 05:12:12.041170 24918 raft_consensus.cc:2229] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:12:12.041517 24918 raft_consensus.cc:2258] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:12:12.043020 24918 ts_tablet_manager.cc:1905] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:12:12.045706 24919 tablet_replica.cc:330] T dfb1250196c34eee969d845ec4edaec4 P 18fd5bc0291745f7874ea50a73a243b3: stopping tablet replica
I20260807 05:12:12.045948 24918 ts_tablet_manager.cc:1918] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:12:12.046057 24918 log.cc:1196] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/wal/wals/dfb1250196c34eee969d845ec4edaec4
I20260807 05:12:12.046267 24919 raft_consensus.cc:2229] T dfb1250196c34eee969d845ec4edaec4 P 18fd5bc0291745f7874ea50a73a243b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:12:12.046458 24918 ts_tablet_manager.cc:1939] T dfb1250196c34eee969d845ec4edaec4 P 0d4b47e627f24cef9e22a5c6bf04bb05: Deleting consensus metadata
I20260807 05:12:12.046715 24919 raft_consensus.cc:2258] T dfb1250196c34eee969d845ec4edaec4 P 18fd5bc0291745f7874ea50a73a243b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:12:12.047681 24361 catalog_manager.cc:4841] TS 0d4b47e627f24cef9e22a5c6bf04bb05 (127.8.16.66:46191): tablet dfb1250196c34eee969d845ec4edaec4 (table test-table [id=f7f24e8e32bd41d1b06fd8b17d56a14c]) successfully deleted
W20260807 05:12:12.048009 24362 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.16.65:38749 (error 108)
I20260807 05:12:12.048290 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 24595
I20260807 05:12:12.049908 24919 ts_tablet_manager.cc:1905] T dfb1250196c34eee969d845ec4edaec4 P 18fd5bc0291745f7874ea50a73a243b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:12:12.050734 24890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:12.050476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:38749 (local address 127.0.0.1:47030)
0807 05:12:12.050580 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:12.050654 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.65:38749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260807 05:12:12.052292 24362 catalog_manager.cc:4584] TS 15a96e9746cf467d93b56735f7572af5 (127.8.16.65:38749): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dfb1250196c34eee969d845ec4edaec4: Network error: Client connection negotiation failed: client connection to 127.8.16.65:38749: connect: Connection refused (error 111)
I20260807 05:12:12.052651 24919 ts_tablet_manager.cc:1918] T dfb1250196c34eee969d845ec4edaec4 P 18fd5bc0291745f7874ea50a73a243b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:12:12.052733 24919 log.cc:1196] T dfb1250196c34eee969d845ec4edaec4 P 18fd5bc0291745f7874ea50a73a243b3: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/dfb1250196c34eee969d845ec4edaec4
I20260807 05:12:12.053002 24919 ts_tablet_manager.cc:1939] T dfb1250196c34eee969d845ec4edaec4 P 18fd5bc0291745f7874ea50a73a243b3: Deleting consensus metadata
I20260807 05:12:12.054184 24360 catalog_manager.cc:4841] TS 18fd5bc0291745f7874ea50a73a243b3 (127.8.16.67:38607): tablet dfb1250196c34eee969d845ec4edaec4 (table test-table [id=f7f24e8e32bd41d1b06fd8b17d56a14c]) successfully deleted
I20260807 05:12:12.055588 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 24739
I20260807 05:12:12.062181 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 24343
I20260807 05:12:32.068595 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:41973
--webserver_interface=127.8.16.126
--webserver_port=33345
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:41973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:12:32.178715 24921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:12:32.178969 24921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:12:32.179008 24921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:12:32.182942 24921 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 05:12:32.182997 24921 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 05:12:32.183012 24921 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 05:12:32.183024 24921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:12:32.183041 24921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:12:32.183058 24921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:12:32.183207 24921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:12:32.183269 24921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:12:32.188547 24921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:41973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:41973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=33345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:12:32.189611 24921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:12:32.190598 24921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:12:32.203419 24928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:12:32.203373 24921 server_base.cc:1034] running on GCE node
W20260807 05:12:32.203372 24930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:12:32.203408 24927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:12:32.204211 24921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:12:32.205334 24921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:12:32.206485 24921 hybrid_clock.cc:648] HybridClock initialized: now 1786079552206466 us; error 34 us; skew 500 ppm
Aug 07 05:12:32 dist-test-slave-49h3 krb5kdc[24328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079552, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:12:32.209156 24921 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:12:32.210045 24921 webserver.cc:460] Webserver started at http://127.8.16.126:33345/ using document root <none> and password file <none>
I20260807 05:12:32.210572 24921 fs_manager.cc:359] Metadata directory not provided
I20260807 05:12:32.210662 24921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:12:32.214365 24921 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:12:32.216513 24939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:32.217729 24921 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:12:32.217852 24921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "d412a46cca7143a7a1aad6a57dc4ba18"
format_stamp: "Formatted at 2026-08-07 05:12:11 on dist-test-slave-49h3"
server_key: "0dec8bb2e91febe04c8884d03cdf8989"
server_key_iv: "6795e39dce447ddf59fdbb7d66e260fa"
server_key_version: "encryptionkey@0"
I20260807 05:12:32.218432 24921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:12:32.252346 24921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:12:32.255628 24921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:12:32.255826 24921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:12:32.262943 24921 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:41973
I20260807 05:12:32.263218 24997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:41973 every 8 connection(s)
I20260807 05:12:32.264024 24921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:12:32.266081 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 24921
I20260807 05:12:32.266562 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:38749
--local_ip_for_outbound_sockets=127.8.16.65
--tserver_master_addrs=127.8.16.126:41973
--webserver_port=33597
--webserver_interface=127.8.16.65
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:12:32.269039 24998 sys_catalog.cc:263] Verifying existing consensus state
I20260807 05:12:32.270771 24998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18: Bootstrap starting.
I20260807 05:12:32.287304 24998 log.cc:824] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18: Log is configured to *not* fsync() on all Append() calls
I20260807 05:12:32.295722 24998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:12:32.296087 24998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18: Bootstrap complete.
I20260807 05:12:32.299913 24998 raft_consensus.cc:348] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41973 } }
I20260807 05:12:32.300294 24998 raft_consensus.cc:729] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d412a46cca7143a7a1aad6a57dc4ba18, State: Initialized, Role: FOLLOWER
I20260807 05:12:32.300743 24998 consensus_queue.cc:260] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41973 } }
I20260807 05:12:32.300860 24998 raft_consensus.cc:388] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:12:32.300926 24998 raft_consensus.cc:482] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:12:32.301013 24998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:12:32.302884 24998 raft_consensus.cc:504] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41973 } }
I20260807 05:12:32.303181 24998 leader_election.cc:302] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d412a46cca7143a7a1aad6a57dc4ba18; no voters:
I20260807 05:12:32.303454 24998 leader_election.cc:288] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 05:12:32.303546 25002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 05:12:32.303773 25002 raft_consensus.cc:686] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 2 LEADER]: Becoming Leader. State: Replica: d412a46cca7143a7a1aad6a57dc4ba18, State: Running, Role: LEADER
I20260807 05:12:32.304113 25002 consensus_queue.cc:237] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41973 } }
I20260807 05:12:32.304634 24998 sys_catalog.cc:564] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:12:32.305335 25003 sys_catalog.cc:455] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41973 } } }
I20260807 05:12:32.305516 25003 sys_catalog.cc:458] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [sys.catalog]: This master's current role is: LEADER
I20260807 05:12:32.305764 25003 sys_catalog.cc:455] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d412a46cca7143a7a1aad6a57dc4ba18. Latest consensus state: current_term: 2 leader_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41973 } } }
I20260807 05:12:32.305881 25003 sys_catalog.cc:458] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [sys.catalog]: This master's current role is: LEADER
I20260807 05:12:32.307017 25010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:12:32.310873 25010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:12:32.311388 25010 catalog_manager.cc:1259] Loaded cluster ID: 3985e8fca84d40a7a434a40437e08ee3
I20260807 05:12:32.311434 25010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:12:32.313431 25010 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:12:32.314005 25010 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18: Loaded TSK: 0
I20260807 05:12:32.322579 25010 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18: Generated new TSK 1
I20260807 05:12:32.322826 25010 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 05:12:32.403939 25000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:12:32.404292 25000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:12:32.404359 25000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:12:32.410316 25000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:12:32.410403 25000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:12:32.410490 25000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:12:32.418175 25000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:38749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=33597
--tserver_master_addrs=127.8.16.126:41973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:12:32.419427 25000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:12:32.420730 25000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:12:32.431013 25029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:12:32.431128 25028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:12:32.431010 25032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:12:32.431533 25000 server_base.cc:1034] running on GCE node
I20260807 05:12:32.431991 25000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:12:32.432770 25000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:12:32.433959 25000 hybrid_clock.cc:648] HybridClock initialized: now 1786079552433929 us; error 41 us; skew 500 ppm
Aug 07 05:12:32 dist-test-slave-49h3 krb5kdc[24328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079552, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:12:32.437268 25000 init.cc:377] Logged in from keytab as kudu/127.8.16.65@KRBTEST.COM (short username kudu)
I20260807 05:12:32.438380 25000 webserver.cc:460] Webserver started at http://127.8.16.65:33597/ using document root <none> and password file <none>
I20260807 05:12:32.439064 25000 fs_manager.cc:359] Metadata directory not provided
I20260807 05:12:32.439170 25000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:12:32.444005 25000 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260807 05:12:32.446206 25039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:32.447602 25000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:12:32.447724 25000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "15a96e9746cf467d93b56735f7572af5"
format_stamp: "Formatted at 2026-08-07 05:12:11 on dist-test-slave-49h3"
server_key: "eb753fe2a4295f32ddaee063cf31770b"
server_key_iv: "91435cd66ef56cb135cf478cb19d5ea2"
server_key_version: "encryptionkey@0"
I20260807 05:12:32.448292 25000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:12:32.518847 25000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:12:32.522063 25000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:12:32.522274 25000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:12:32.522886 25000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:12:32.523944 25046 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 05:12:32.527979 25000 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 05:12:32.528047 25000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260807 05:12:32.528110 25000 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 05:12:32.529711 25000 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 05:12:32.529758 25000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260807 05:12:32.530010 25046 tablet_bootstrap.cc:492] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5: Bootstrap starting.
I20260807 05:12:32.539088 25000 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:38749
I20260807 05:12:32.539194 25168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:38749 every 8 connection(s)
I20260807 05:12:32.540004 25000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:12:32.545252 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 25000
I20260807 05:12:32.545821 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:46191
--local_ip_for_outbound_sockets=127.8.16.66
--tserver_master_addrs=127.8.16.126:41973
--webserver_port=36861
--webserver_interface=127.8.16.66
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:12:32.547998 25046 log.cc:824] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5: Log is configured to *not* fsync() on all Append() calls
Aug 07 05:12:32 dist-test-slave-49h3 krb5kdc[24328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079552, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:12:32.555960 25046 tablet_bootstrap.cc:492] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:12:32.556398 25046 tablet_bootstrap.cc:492] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5: Bootstrap complete.
I20260807 05:12:32.557186 25173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:32.542142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.65:60979)
0807 05:12:32.542613 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:32.542655 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:12:32.543418 (+ 763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:12:32.544467 (+ 1049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:12:32.544475 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:12:32.544877 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:12:32.545312 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:32.545328 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.546764 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.546767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:32.547724 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:32.547736 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.548690 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.548694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:32.548880 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:32.549527 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:12:32.549547 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:12:32.551317 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:12:32.553831 (+ 2514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:12:32.553837 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:12:32.553848 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:12:32.554172 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:12:32.554555 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:12:32.554558 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:12:32.554560 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:12:32.554731 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:12:32.555440 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:12:32.555446 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:12:32.555741 (+ 295us) client_negotiation.cc:769] Sending connection context
0807 05:12:32.555934 (+ 193us) client_negotiation.cc:240] Negotiation successful
0807 05:12:32.556268 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":141,"threads_started":1}
I20260807 05:12:32.557632 25046 ts_tablet_manager.cc:1397] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5: Time spent bootstrapping tablet: real 0.028s user 0.015s sys 0.012s
I20260807 05:12:32.557777 25172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:32.542035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:60979 (local address 127.8.16.126:41973)
0807 05:12:32.542388 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:32.542409 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:32.542778 (+ 369us) server_negotiation.cc:407] Connection header received
0807 05:12:32.543819 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:32.543829 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:32.544101 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:32.544396 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:12:32.545533 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.546624 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.547858 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.548542 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.548706 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:32.551526 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:12:32.551533 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:12:32.551549 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:12:32.551585 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:12:32.553708 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:32.554290 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:32.554296 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:32.554302 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:32.554456 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:32.554825 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:32.554828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:32.554830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:32.555316 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:12:32.555455 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:32.556189 (+ 734us) server_negotiation.cc:299] Negotiation successful
0807 05:12:32.556560 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":111,"threads_started":1}
I20260807 05:12:32.560205 25169 heartbeater.cc:347] Connected to a master server at 127.8.16.126:41973
I20260807 05:12:32.560501 25169 heartbeater.cc:464] Registering TS with master...
I20260807 05:12:32.560230 25046 raft_consensus.cc:348] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } }
I20260807 05:12:32.560670 25046 raft_consensus.cc:729] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15a96e9746cf467d93b56735f7572af5, State: Initialized, Role: FOLLOWER
I20260807 05:12:32.561025 25169 heartbeater.cc:511] Master 127.8.16.126:41973 requested a full tablet report, sending...
I20260807 05:12:32.561065 25046 consensus_queue.cc:260] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } }
I20260807 05:12:32.561934 25046 ts_tablet_manager.cc:1428] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260807 05:12:32.563460 24958 ts_manager.cc:194] Registered new tserver with Master: 15a96e9746cf467d93b56735f7572af5 (127.8.16.65:38749)
I20260807 05:12:32.566727 24958 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:60979
I20260807 05:12:32.569285 25169 heartbeater.cc:503] Master 127.8.16.126:41973 was elected leader, sending a full tablet report...
Aug 07 05:12:32 dist-test-slave-49h3 krb5kdc[24328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079552, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for kudu/127.8.16.65@KRBTEST.COM
I20260807 05:12:32.575686 25177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:32.566186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:38749 (local address 127.0.0.1:34262)
0807 05:12:32.566863 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:32.566906 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 05:12:32.567038 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:12:32.567563 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:12:32.567569 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:12:32.567926 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:12:32.568028 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:32.568035 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.568983 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.568985 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:32.569752 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:32.569763 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.570447 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.570450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:32.570562 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:32.571124 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:12:32.571139 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:12:32.572396 (+ 1257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:12:32.574091 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:12:32.574100 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:12:32.574105 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:12:32.574322 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:12:32.574590 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:12:32.574593 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:12:32.574595 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:12:32.574677 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:12:32.575089 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:12:32.575095 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:12:32.575207 (+ 112us) client_negotiation.cc:769] Sending connection context
0807 05:12:32.575321 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 05:12:32.575509 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":91,"threads_started":1}
I20260807 05:12:32.575716 25178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:32.566262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34262 (local address 127.8.16.65:38749)
0807 05:12:32.566605 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:32.566618 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:32.567000 (+ 382us) server_negotiation.cc:407] Connection header received
0807 05:12:32.567188 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:32.567193 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:32.567380 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:32.567499 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:12:32.568152 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.568872 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.569862 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.570330 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.570464 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:32.572525 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:12:32.572534 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:12:32.572543 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:12:32.572564 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:12:32.573981 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:32.574426 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:32.574430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:32.574432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:32.574506 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:32.574756 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:32.574759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:32.574761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:32.574974 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:12:32.575090 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:32.575416 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 05:12:32.575563 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":260,"thread_start_us":93,"threads_started":1}
I20260807 05:12:32.576685 25094 tablet_service.cc:1514] Processing DeleteTablet for tablet dfb1250196c34eee969d845ec4edaec4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:12:12 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:34262
I20260807 05:12:32.577670 25179 tablet_replica.cc:330] stopping tablet replica
I20260807 05:12:32.577937 25179 raft_consensus.cc:2229] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:12:32.578150 25179 raft_consensus.cc:2258] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:12:32.579000 25179 ts_tablet_manager.cc:1905] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:12:32.582382 25179 ts_tablet_manager.cc:1918] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:12:32.582472 25179 log.cc:1196] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/wal/wals/dfb1250196c34eee969d845ec4edaec4
I20260807 05:12:32.582711 25179 ts_tablet_manager.cc:1939] T dfb1250196c34eee969d845ec4edaec4 P 15a96e9746cf467d93b56735f7572af5: Deleting consensus metadata
I20260807 05:12:32.584354 24944 catalog_manager.cc:4841] TS 15a96e9746cf467d93b56735f7572af5 (127.8.16.65:38749): tablet dfb1250196c34eee969d845ec4edaec4 (table test-table [id=f7f24e8e32bd41d1b06fd8b17d56a14c]) successfully deleted
W20260807 05:12:32.658545 25174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:12:32.658792 25174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:12:32.658838 25174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:12:32.662892 25174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:12:32.662946 25174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:12:32.663015 25174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:12:32.668179 25174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:46191
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=36861
--tserver_master_addrs=127.8.16.126:41973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:12:32.669191 25174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:12:32.670212 25174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:12:32.679267 25187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:12:32.679283 25186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:12:32.679271 25189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:12:32.679478 25174 server_base.cc:1034] running on GCE node
I20260807 05:12:32.679980 25174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:12:32.680589 25174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:12:32.681738 25174 hybrid_clock.cc:648] HybridClock initialized: now 1786079552681719 us; error 34 us; skew 500 ppm
Aug 07 05:12:32 dist-test-slave-49h3 krb5kdc[24328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079552, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:12:32.684310 25174 init.cc:377] Logged in from keytab as kudu/127.8.16.66@KRBTEST.COM (short username kudu)
I20260807 05:12:32.685151 25174 webserver.cc:460] Webserver started at http://127.8.16.66:36861/ using document root <none> and password file <none>
I20260807 05:12:32.685716 25174 fs_manager.cc:359] Metadata directory not provided
I20260807 05:12:32.685797 25174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:12:32.689368 25174 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:12:32.691296 25196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:32.692386 25174 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:12:32.692485 25174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "0d4b47e627f24cef9e22a5c6bf04bb05"
format_stamp: "Formatted at 2026-08-07 05:12:11 on dist-test-slave-49h3"
server_key: "c6f0933fe8a13eb007d0521140c1253b"
server_key_iv: "0d57ecc52845f1f05c8c2350331cc3ec"
server_key_version: "encryptionkey@0"
I20260807 05:12:32.692978 25174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:12:32.706635 25174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:12:32.709479 25174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:12:32.709657 25174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:12:32.710180 25174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:12:32.710952 25174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:12:32.710997 25174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:32.711035 25174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:12:32.711054 25174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:32.719177 25174 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:46191
I20260807 05:12:32.719226 25316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:46191 every 8 connection(s)
I20260807 05:12:32.720099 25174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:12:32.723075 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 25174
I20260807 05:12:32.723475 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:38607
--local_ip_for_outbound_sockets=127.8.16.67
--tserver_master_addrs=127.8.16.126:41973
--webserver_port=44625
--webserver_interface=127.8.16.67
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 05:12:32 dist-test-slave-49h3 krb5kdc[24328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079552, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:12:32.733156 25172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:32.721796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:38151 (local address 127.8.16.126:41973)
0807 05:12:32.721919 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:32.721923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:32.723060 (+ 1137us) server_negotiation.cc:407] Connection header received
0807 05:12:32.724008 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:32.724012 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:32.724073 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:32.724142 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:12:32.725248 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.725826 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.726918 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.727145 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.727234 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:32.729858 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:12:32.729861 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:12:32.729863 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:12:32.729890 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:12:32.731149 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:32.731712 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:32.731715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:32.731716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:32.731782 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:32.732137 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:32.732139 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:32.732140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:32.732290 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:12:32.732356 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:32.732930 (+ 574us) server_negotiation.cc:299] Negotiation successful
0807 05:12:32.733035 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 05:12:32.734305 25319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:32.722187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.66:38151)
0807 05:12:32.722914 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:32.722966 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 05:12:32.723816 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:12:32.724325 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:12:32.724332 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:12:32.724664 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:12:32.725098 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:32.725111 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.725942 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.725945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:32.726803 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:32.726813 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.727277 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.727281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:32.727432 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:32.727967 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:12:32.727982 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:12:32.729691 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:12:32.731262 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:12:32.731268 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:12:32.731283 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:12:32.731626 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:12:32.731860 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:12:32.731863 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:12:32.731865 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:12:32.732044 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:12:32.732379 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:12:32.732386 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:12:32.732696 (+ 310us) client_negotiation.cc:769] Sending connection context
0807 05:12:32.732915 (+ 219us) client_negotiation.cc:240] Negotiation successful
0807 05:12:32.733258 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":109,"threads_started":1}
I20260807 05:12:32.735394 25317 heartbeater.cc:347] Connected to a master server at 127.8.16.126:41973
I20260807 05:12:32.735599 25317 heartbeater.cc:464] Registering TS with master...
I20260807 05:12:32.736040 25317 heartbeater.cc:511] Master 127.8.16.126:41973 requested a full tablet report, sending...
I20260807 05:12:32.736964 24958 ts_manager.cc:194] Registered new tserver with Master: 0d4b47e627f24cef9e22a5c6bf04bb05 (127.8.16.66:46191)
I20260807 05:12:32.737743 24958 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:38151
W20260807 05:12:32.832563 25320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:12:32.832819 25320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:12:32.832862 25320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:12:32.836798 25320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:12:32.836858 25320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:12:32.836917 25320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:12:32.842121 25320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:38607
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=44625
--tserver_master_addrs=127.8.16.126:41973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:12:32.843135 25320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:12:32.844105 25320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:12:32.853369 25329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:12:32.853397 25326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:12:32.853377 25327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:12:32.853628 25320 server_base.cc:1034] running on GCE node
I20260807 05:12:32.854137 25320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:12:32.854756 25320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:12:32.855924 25320 hybrid_clock.cc:648] HybridClock initialized: now 1786079552855906 us; error 29 us; skew 500 ppm
Aug 07 05:12:32 dist-test-slave-49h3 krb5kdc[24328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079552, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:12:32.858708 25320 init.cc:377] Logged in from keytab as kudu/127.8.16.67@KRBTEST.COM (short username kudu)
I20260807 05:12:32.859576 25320 webserver.cc:460] Webserver started at http://127.8.16.67:44625/ using document root <none> and password file <none>
I20260807 05:12:32.860154 25320 fs_manager.cc:359] Metadata directory not provided
I20260807 05:12:32.860255 25320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:12:32.864070 25320 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260807 05:12:32.866209 25336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:32.867390 25320 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:12:32.867499 25320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "18fd5bc0291745f7874ea50a73a243b3"
format_stamp: "Formatted at 2026-08-07 05:12:11 on dist-test-slave-49h3"
server_key: "80248c25c3a6cc7cd49e8699e9e11837"
server_key_iv: "710a7dbc29458e275700e26f5a3a88bb"
server_key_version: "encryptionkey@0"
I20260807 05:12:32.868021 25320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:12:32.882072 25320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:12:32.885005 25320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:12:32.885198 25320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:12:32.885838 25320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:12:32.886678 25320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:12:32.886725 25320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:32.886777 25320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:12:32.886806 25320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:32.896132 25320 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:38607
I20260807 05:12:32.896164 25462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:38607 every 8 connection(s)
I20260807 05:12:32.897217 25320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:12:32.899726 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 25320
Aug 07 05:12:32 dist-test-slave-49h3 krb5kdc[24328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079552, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:12:32.908716 25467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:32.900362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53996 (local address 127.8.16.126:41973)
0807 05:12:32.900617 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:32.900622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:32.900916 (+ 294us) server_negotiation.cc:407] Connection header received
0807 05:12:32.901045 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:32.901048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:32.901093 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:32.901173 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:12:32.902124 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.902676 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.903621 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.903847 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.903920 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:32.905376 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:12:32.905379 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:12:32.905381 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:12:32.905401 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:12:32.907014 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:32.907457 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:32.907460 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:32.907461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:32.907543 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:32.907842 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:32.907845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:32.907847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:32.908061 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:12:32.908276 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:32.908409 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 05:12:32.908527 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":80,"threads_started":1}
I20260807 05:12:32.911041 25172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:32.898968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:45981 (local address 127.8.16.126:41973)
0807 05:12:32.899104 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:32.899107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:32.899940 (+ 833us) server_negotiation.cc:407] Connection header received
0807 05:12:32.901492 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:32.901495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:32.901530 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:32.901588 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:12:32.902843 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.903395 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.904778 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.905013 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.905090 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:32.907703 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:12:32.907705 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:12:32.907706 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:12:32.907722 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:12:32.909048 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:32.909591 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:32.909595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:32.909596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:32.909677 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:32.909981 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:32.909984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:32.909986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:32.910148 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:12:32.910233 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:32.910782 (+ 549us) server_negotiation.cc:299] Negotiation successful
0807 05:12:32.910893 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:12:32.911835 25465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:32.899321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.67:45981)
0807 05:12:32.899782 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:32.899832 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 05:12:32.901253 (+ 1421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:12:32.901782 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:12:32.901791 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:12:32.902243 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:12:32.902709 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:32.902723 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.903511 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.903515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:32.904663 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:32.904675 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.905159 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.905164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:32.905364 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:32.905882 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:12:32.905898 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:12:32.907572 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:12:32.909150 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:12:32.909155 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:12:32.909178 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:12:32.909494 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:12:32.909758 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:12:32.909760 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:12:32.909762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:12:32.909889 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:12:32.910246 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:12:32.910253 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:12:32.910543 (+ 290us) client_negotiation.cc:769] Sending connection context
0807 05:12:32.910708 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 05:12:32.911025 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":115,"threads_started":1}
I20260807 05:12:32.912822 25463 heartbeater.cc:347] Connected to a master server at 127.8.16.126:41973
I20260807 05:12:32.913041 25463 heartbeater.cc:464] Registering TS with master...
I20260807 05:12:32.913547 25463 heartbeater.cc:511] Master 127.8.16.126:41973 requested a full tablet report, sending...
I20260807 05:12:32.914531 24958 ts_manager.cc:194] Registered new tserver with Master: 18fd5bc0291745f7874ea50a73a243b3 (127.8.16.67:38607)
I20260807 05:12:32.915057 24958 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:45981
I20260807 05:12:32.922135 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:12:32.926077 25172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:32.922698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53998 (local address 127.8.16.126:41973)
0807 05:12:32.922812 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:32.922816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:32.923012 (+ 196us) server_negotiation.cc:407] Connection header received
0807 05:12:32.923180 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:32.923183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:32.923281 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:32.923367 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:12:32.923611 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.924108 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.924947 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.925174 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.925260 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:32.925583 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:12:32.925703 (+ 120us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 05:12:32.925942 (+ 239us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53998: token expired
Metrics: {"server-negotiator.queue_time_us":39}
W20260807 05:12:32.926136 25172 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53998: token expired
W20260807 05:12:32.926410 24885 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.16.126:41973)
I20260807 05:12:32.933038 25172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:32.926716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54014 (local address 127.8.16.126:41973)
0807 05:12:32.926810 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:32.926814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:32.926851 (+ 37us) server_negotiation.cc:407] Connection header received
0807 05:12:32.926968 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:32.926970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:32.927010 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:32.927071 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:12:32.927673 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.928137 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.928879 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.929095 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.929177 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:32.930441 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:12:32.930445 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:12:32.930447 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:12:32.930472 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:12:32.931551 (+ 1079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:32.931958 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:32.931962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:32.931964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:32.932040 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:32.932283 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:32.932286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:32.932287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:32.932449 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:12:32.932575 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:32.932781 (+ 206us) server_negotiation.cc:299] Negotiation successful
0807 05:12:32.932886 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 05:12:32.935412 24958 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54014:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:12:32.938535 24958 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:12:32.945858 25094 tablet_service.cc:1467] Processing CreateTablet for tablet 97e1209932ce474ea21e384e366c8322 (DEFAULT_TABLE table=test-table [id=4f4ba4e020d14b2aa7ebb774182acf48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:12:32.946511 25094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97e1209932ce474ea21e384e366c8322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:12:32.949036 25046 tablet_bootstrap.cc:492] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5: Bootstrap starting.
I20260807 05:12:32.950055 25046 tablet_bootstrap.cc:654] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5: Neither blocks nor log segments found. Creating new log.
I20260807 05:12:32.950650 25177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:32.944646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:46191 (local address 127.0.0.1:46080)
0807 05:12:32.944887 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:32.944903 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:12:32.944975 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:12:32.945965 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:12:32.945968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:12:32.945984 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:12:32.946106 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:32.946114 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.947737 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.947740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:32.948885 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:32.948892 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.950278 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.950281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:32.950357 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:32.950375 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 05:12:32.950417 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 05:12:32.950520 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":37}
I20260807 05:12:32.950798 25475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:32.945165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50974 (local address 127.8.16.67:38607)
0807 05:12:32.945503 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:32.945514 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:32.945757 (+ 243us) server_negotiation.cc:407] Connection header received
0807 05:12:32.945828 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:32.945831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:32.945986 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:32.946203 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:12:32.946528 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.947421 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.949014 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.949875 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.949970 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:32.950130 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:32.950477 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 05:12:32.950661 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":124,"threads_started":1}
I20260807 05:12:32.951022 25474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:32.944930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:38607 (local address 127.0.0.1:50974)
0807 05:12:32.945542 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:32.945555 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:12:32.945663 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:12:32.946343 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:12:32.946345 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:12:32.946359 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:12:32.946423 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:32.946427 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.947548 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.947551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:32.948859 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:32.948872 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.950197 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.950201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:32.950291 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:32.950338 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 05:12:32.950414 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 05:12:32.950900 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"mutex_wait_us":16,"thread_start_us":112,"threads_started":1}
I20260807 05:12:32.951397 25473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:32.944835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46080 (local address 127.8.16.66:46191)
0807 05:12:32.945492 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:32.945503 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:32.945528 (+ 25us) server_negotiation.cc:407] Connection header received
0807 05:12:32.945603 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:32.945608 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:32.945813 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:32.945979 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:12:32.946273 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.947587 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.949014 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:32.950074 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:32.950639 (+ 565us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:32.951130 (+ 491us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:32.951182 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 05:12:32.951267 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":177,"threads_started":1}
I20260807 05:12:32.951484 25046 tablet_bootstrap.cc:492] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5: No bootstrap required, opened a new log
I20260807 05:12:32.951574 25046 ts_tablet_manager.cc:1397] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 05:12:32.952033 25046 raft_consensus.cc:348] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } }
I20260807 05:12:32.952155 25046 raft_consensus.cc:374] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:12:32.952195 25046 raft_consensus.cc:729] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15a96e9746cf467d93b56735f7572af5, State: Initialized, Role: FOLLOWER
I20260807 05:12:32.952318 25046 consensus_queue.cc:260] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } }
I20260807 05:12:32.952555 25046 ts_tablet_manager.cc:1428] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:12:32.954015 25383 tablet_service.cc:1467] Processing CreateTablet for tablet 97e1209932ce474ea21e384e366c8322 (DEFAULT_TABLE table=test-table [id=4f4ba4e020d14b2aa7ebb774182acf48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:12:32.954015 25247 tablet_service.cc:1467] Processing CreateTablet for tablet 97e1209932ce474ea21e384e366c8322 (DEFAULT_TABLE table=test-table [id=4f4ba4e020d14b2aa7ebb774182acf48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:12:32.955120 25383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97e1209932ce474ea21e384e366c8322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:12:32.955121 25247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97e1209932ce474ea21e384e366c8322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:12:32.963233 25478 tablet_bootstrap.cc:492] T 97e1209932ce474ea21e384e366c8322 P 18fd5bc0291745f7874ea50a73a243b3: Bootstrap starting.
I20260807 05:12:32.963233 25477 tablet_bootstrap.cc:492] T 97e1209932ce474ea21e384e366c8322 P 0d4b47e627f24cef9e22a5c6bf04bb05: Bootstrap starting.
I20260807 05:12:32.966018 25477 tablet_bootstrap.cc:654] T 97e1209932ce474ea21e384e366c8322 P 0d4b47e627f24cef9e22a5c6bf04bb05: Neither blocks nor log segments found. Creating new log.
I20260807 05:12:32.966005 25478 tablet_bootstrap.cc:654] T 97e1209932ce474ea21e384e366c8322 P 18fd5bc0291745f7874ea50a73a243b3: Neither blocks nor log segments found. Creating new log.
I20260807 05:12:32.966919 25477 log.cc:824] T 97e1209932ce474ea21e384e366c8322 P 0d4b47e627f24cef9e22a5c6bf04bb05: Log is configured to *not* fsync() on all Append() calls
I20260807 05:12:32.966919 25478 log.cc:824] T 97e1209932ce474ea21e384e366c8322 P 18fd5bc0291745f7874ea50a73a243b3: Log is configured to *not* fsync() on all Append() calls
I20260807 05:12:32.968904 25477 tablet_bootstrap.cc:492] T 97e1209932ce474ea21e384e366c8322 P 0d4b47e627f24cef9e22a5c6bf04bb05: No bootstrap required, opened a new log
I20260807 05:12:32.968915 25478 tablet_bootstrap.cc:492] T 97e1209932ce474ea21e384e366c8322 P 18fd5bc0291745f7874ea50a73a243b3: No bootstrap required, opened a new log
I20260807 05:12:32.969089 25478 ts_tablet_manager.cc:1397] T 97e1209932ce474ea21e384e366c8322 P 18fd5bc0291745f7874ea50a73a243b3: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 05:12:32.969105 25477 ts_tablet_manager.cc:1397] T 97e1209932ce474ea21e384e366c8322 P 0d4b47e627f24cef9e22a5c6bf04bb05: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 05:12:32.971972 25478 raft_consensus.cc:348] T 97e1209932ce474ea21e384e366c8322 P 18fd5bc0291745f7874ea50a73a243b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } }
I20260807 05:12:32.971975 25477 raft_consensus.cc:348] T 97e1209932ce474ea21e384e366c8322 P 0d4b47e627f24cef9e22a5c6bf04bb05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } }
I20260807 05:12:32.972211 25477 raft_consensus.cc:374] T 97e1209932ce474ea21e384e366c8322 P 0d4b47e627f24cef9e22a5c6bf04bb05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:12:32.972215 25478 raft_consensus.cc:374] T 97e1209932ce474ea21e384e366c8322 P 18fd5bc0291745f7874ea50a73a243b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:12:32.972245 25477 raft_consensus.cc:729] T 97e1209932ce474ea21e384e366c8322 P 0d4b47e627f24cef9e22a5c6bf04bb05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d4b47e627f24cef9e22a5c6bf04bb05, State: Initialized, Role: FOLLOWER
I20260807 05:12:32.972275 25478 raft_consensus.cc:729] T 97e1209932ce474ea21e384e366c8322 P 18fd5bc0291745f7874ea50a73a243b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18fd5bc0291745f7874ea50a73a243b3, State: Initialized, Role: FOLLOWER
I20260807 05:12:32.972659 25477 consensus_queue.cc:260] T 97e1209932ce474ea21e384e366c8322 P 0d4b47e627f24cef9e22a5c6bf04bb05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } }
I20260807 05:12:32.972653 25478 consensus_queue.cc:260] T 97e1209932ce474ea21e384e366c8322 P 18fd5bc0291745f7874ea50a73a243b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } }
I20260807 05:12:32.973320 25463 heartbeater.cc:503] Master 127.8.16.126:41973 was elected leader, sending a full tablet report...
I20260807 05:12:32.973582 25317 heartbeater.cc:503] Master 127.8.16.126:41973 was elected leader, sending a full tablet report...
I20260807 05:12:32.973654 25477 ts_tablet_manager.cc:1428] T 97e1209932ce474ea21e384e366c8322 P 0d4b47e627f24cef9e22a5c6bf04bb05: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 05:12:32.973662 25478 ts_tablet_manager.cc:1428] T 97e1209932ce474ea21e384e366c8322 P 18fd5bc0291745f7874ea50a73a243b3: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
W20260807 05:12:33.041199 25171 tablet.cc:2369] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:12:33.082746 25176 raft_consensus.cc:482] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:12:33.082998 25176 raft_consensus.cc:504] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } }
I20260807 05:12:33.084158 25176 leader_election.cc:288] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d4b47e627f24cef9e22a5c6bf04bb05 (127.8.16.66:46191), 18fd5bc0291745f7874ea50a73a243b3 (127.8.16.67:38607)
I20260807 05:12:33.088006 25473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:33.084556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:43769 (local address 127.8.16.66:46191)
0807 05:12:33.084746 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:33.084755 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:33.084815 (+ 60us) server_negotiation.cc:407] Connection header received
0807 05:12:33.085000 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:33.085003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:33.085051 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:33.085120 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:12:33.085401 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:33.085931 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:33.087031 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:33.087436 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:33.087526 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:33.087583 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:33.087814 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 05:12:33.087884 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260807 05:12:33.088007 25475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:33.084547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:34657 (local address 127.8.16.67:38607)
0807 05:12:33.084682 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:33.084686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:33.084882 (+ 196us) server_negotiation.cc:407] Connection header received
0807 05:12:33.085021 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:33.085023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:33.085068 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:33.085127 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:12:33.085404 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:33.085928 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:33.087030 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:33.087425 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:33.087516 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:33.087579 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:33.087812 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 05:12:33.087878 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 05:12:33.088066 25484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:33.084427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:38607 (local address 127.8.16.65:34657)
0807 05:12:33.084796 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:33.084807 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:12:33.084912 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:12:33.085151 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:12:33.085154 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:12:33.085183 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:12:33.085308 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:33.085314 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:33.086033 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:33.086037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:33.086917 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:33.086926 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:33.087538 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:33.087542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:33.087684 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:33.087722 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 05:12:33.087785 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 05:12:33.087870 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":129,"spinlock_wait_cycles":14592,"thread_start_us":98,"threads_started":1}
I20260807 05:12:33.088068 25483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:33.084353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:46191 (local address 127.8.16.65:43769)
0807 05:12:33.084720 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:33.084735 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:12:33.084875 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:12:33.085152 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:12:33.085155 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:12:33.085190 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:12:33.085306 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:33.085311 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:33.086036 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:33.086039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:33.086916 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:33.086925 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:33.087546 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:33.087550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:33.087681 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:33.087721 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 05:12:33.087786 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 05:12:33.087870 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":33,"spinlock_wait_cycles":5376,"thread_start_us":105,"threads_started":1}
I20260807 05:12:33.088650 25414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97e1209932ce474ea21e384e366c8322" candidate_uuid: "15a96e9746cf467d93b56735f7572af5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18fd5bc0291745f7874ea50a73a243b3" is_pre_election: true
I20260807 05:12:33.088773 25268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97e1209932ce474ea21e384e366c8322" candidate_uuid: "15a96e9746cf467d93b56735f7572af5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" is_pre_election: true
I20260807 05:12:33.088958 25414 raft_consensus.cc:2454] T 97e1209932ce474ea21e384e366c8322 P 18fd5bc0291745f7874ea50a73a243b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15a96e9746cf467d93b56735f7572af5 in term 0.
I20260807 05:12:33.089044 25268 raft_consensus.cc:2454] T 97e1209932ce474ea21e384e366c8322 P 0d4b47e627f24cef9e22a5c6bf04bb05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15a96e9746cf467d93b56735f7572af5 in term 0.
I20260807 05:12:33.089403 25041 leader_election.cc:302] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15a96e9746cf467d93b56735f7572af5, 18fd5bc0291745f7874ea50a73a243b3; no voters:
I20260807 05:12:33.089700 25176 raft_consensus.cc:2781] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:12:33.089779 25176 raft_consensus.cc:482] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:12:33.089812 25176 raft_consensus.cc:3037] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:12:33.090799 25176 raft_consensus.cc:504] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } }
I20260807 05:12:33.091154 25176 leader_election.cc:288] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [CANDIDATE]: Term 1 election: Requested vote from peers 0d4b47e627f24cef9e22a5c6bf04bb05 (127.8.16.66:46191), 18fd5bc0291745f7874ea50a73a243b3 (127.8.16.67:38607)
I20260807 05:12:33.091475 25268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97e1209932ce474ea21e384e366c8322" candidate_uuid: "15a96e9746cf467d93b56735f7572af5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05"
I20260807 05:12:33.091516 25414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97e1209932ce474ea21e384e366c8322" candidate_uuid: "15a96e9746cf467d93b56735f7572af5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18fd5bc0291745f7874ea50a73a243b3"
I20260807 05:12:33.091581 25268 raft_consensus.cc:3037] T 97e1209932ce474ea21e384e366c8322 P 0d4b47e627f24cef9e22a5c6bf04bb05 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:12:33.091624 25414 raft_consensus.cc:3037] T 97e1209932ce474ea21e384e366c8322 P 18fd5bc0291745f7874ea50a73a243b3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:12:33.092566 25268 raft_consensus.cc:2454] T 97e1209932ce474ea21e384e366c8322 P 0d4b47e627f24cef9e22a5c6bf04bb05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15a96e9746cf467d93b56735f7572af5 in term 1.
I20260807 05:12:33.092631 25414 raft_consensus.cc:2454] T 97e1209932ce474ea21e384e366c8322 P 18fd5bc0291745f7874ea50a73a243b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15a96e9746cf467d93b56735f7572af5 in term 1.
I20260807 05:12:33.092839 25042 leader_election.cc:302] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d4b47e627f24cef9e22a5c6bf04bb05, 15a96e9746cf467d93b56735f7572af5; no voters:
I20260807 05:12:33.093024 25176 raft_consensus.cc:2781] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:12:33.093211 25176 raft_consensus.cc:686] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 1 LEADER]: Becoming Leader. State: Replica: 15a96e9746cf467d93b56735f7572af5, State: Running, Role: LEADER
I20260807 05:12:33.093438 25176 consensus_queue.cc:237] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } }
I20260807 05:12:33.096455 24957 catalog_manager.cc:5495] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15a96e9746cf467d93b56735f7572af5 (127.8.16.65). New cstate: current_term: 1 leader_uuid: "15a96e9746cf467d93b56735f7572af5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } health_report { overall_health: HEALTHY } } }
I20260807 05:12:33.141075 25488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:33.137794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34264 (local address 127.8.16.65:38749)
0807 05:12:33.138072 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:33.138079 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:33.138091 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:12:33.138188 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:33.138190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:33.138245 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:33.138341 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:12:33.138563 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:33.139060 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:33.139878 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:33.140087 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:33.140166 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:33.140362 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:12:33.140770 (+ 408us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:12:33.140849 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:33.140891 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 05:12:33.140939 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":105,"threads_started":1}
I20260807 05:12:33.145025 25268 raft_consensus.cc:1261] T 97e1209932ce474ea21e384e366c8322 P 0d4b47e627f24cef9e22a5c6bf04bb05 [term 1 FOLLOWER]: Refusing update from remote peer 15a96e9746cf467d93b56735f7572af5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:12:33.145025 25414 raft_consensus.cc:1261] T 97e1209932ce474ea21e384e366c8322 P 18fd5bc0291745f7874ea50a73a243b3 [term 1 FOLLOWER]: Refusing update from remote peer 15a96e9746cf467d93b56735f7572af5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:12:33.145741 25485 consensus_queue.cc:1035] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:12:33.145920 25176 consensus_queue.cc:1035] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 05:12:33.148154 25464 tablet.cc:2369] T 97e1209932ce474ea21e384e366c8322 P 18fd5bc0291745f7874ea50a73a243b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:12:33.153229 25489 mvcc.cc:205] Tried to move back new op lower bound from 7315781849675460608 to 7315781849473175552. Current Snapshot: MvccSnapshot[applied={T|T < 7315781849675460608}]
I20260807 05:12:33.154098 25490 mvcc.cc:205] Tried to move back new op lower bound from 7315781849675460608 to 7315781849473175552. Current Snapshot: MvccSnapshot[applied={T|T < 7315781849675460608}]
I20260807 05:12:33.160667 24957 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:12:33.160804 24957 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:12:33.162914 24957 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18: Sending DeleteTablet for 3 replicas of tablet 97e1209932ce474ea21e384e366c8322
I20260807 05:12:33.163530 25247 tablet_service.cc:1514] Processing DeleteTablet for tablet 97e1209932ce474ea21e384e366c8322 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:12:33 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:46080
I20260807 05:12:33.163550 25383 tablet_service.cc:1514] Processing DeleteTablet for tablet 97e1209932ce474ea21e384e366c8322 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:12:33 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:50974
I20260807 05:12:33.163604 25094 tablet_service.cc:1514] Processing DeleteTablet for tablet 97e1209932ce474ea21e384e366c8322 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:12:33 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:34262
I20260807 05:12:33.163877 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 25000
I20260807 05:12:33.163951 25498 tablet_replica.cc:330] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5: stopping tablet replica
I20260807 05:12:33.165943 25499 tablet_replica.cc:330] T 97e1209932ce474ea21e384e366c8322 P 0d4b47e627f24cef9e22a5c6bf04bb05: stopping tablet replica
I20260807 05:12:33.166240 25499 raft_consensus.cc:2229] T 97e1209932ce474ea21e384e366c8322 P 0d4b47e627f24cef9e22a5c6bf04bb05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:12:33.166491 25499 raft_consensus.cc:2258] T 97e1209932ce474ea21e384e366c8322 P 0d4b47e627f24cef9e22a5c6bf04bb05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:12:33.166778 25500 tablet_replica.cc:330] T 97e1209932ce474ea21e384e366c8322 P 18fd5bc0291745f7874ea50a73a243b3: stopping tablet replica
I20260807 05:12:33.167069 25500 raft_consensus.cc:2229] T 97e1209932ce474ea21e384e366c8322 P 18fd5bc0291745f7874ea50a73a243b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:12:33.167312 25500 raft_consensus.cc:2258] T 97e1209932ce474ea21e384e366c8322 P 18fd5bc0291745f7874ea50a73a243b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:12:33.168565 25499 ts_tablet_manager.cc:1905] T 97e1209932ce474ea21e384e366c8322 P 0d4b47e627f24cef9e22a5c6bf04bb05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:12:33.168918 25500 ts_tablet_manager.cc:1905] T 97e1209932ce474ea21e384e366c8322 P 18fd5bc0291745f7874ea50a73a243b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:12:33.170768 25499 ts_tablet_manager.cc:1918] T 97e1209932ce474ea21e384e366c8322 P 0d4b47e627f24cef9e22a5c6bf04bb05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:12:33.170852 25499 log.cc:1196] T 97e1209932ce474ea21e384e366c8322 P 0d4b47e627f24cef9e22a5c6bf04bb05: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/wal/wals/97e1209932ce474ea21e384e366c8322
I20260807 05:12:33.171200 25499 ts_tablet_manager.cc:1939] T 97e1209932ce474ea21e384e366c8322 P 0d4b47e627f24cef9e22a5c6bf04bb05: Deleting consensus metadata
I20260807 05:12:33.171608 25500 ts_tablet_manager.cc:1918] T 97e1209932ce474ea21e384e366c8322 P 18fd5bc0291745f7874ea50a73a243b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:12:33.171700 25500 log.cc:1196] T 97e1209932ce474ea21e384e366c8322 P 18fd5bc0291745f7874ea50a73a243b3: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/97e1209932ce474ea21e384e366c8322
I20260807 05:12:33.172022 25500 ts_tablet_manager.cc:1939] T 97e1209932ce474ea21e384e366c8322 P 18fd5bc0291745f7874ea50a73a243b3: Deleting consensus metadata
I20260807 05:12:33.172256 24943 catalog_manager.cc:4841] TS 0d4b47e627f24cef9e22a5c6bf04bb05 (127.8.16.66:46191): tablet 97e1209932ce474ea21e384e366c8322 (table test-table [id=4f4ba4e020d14b2aa7ebb774182acf48]) successfully deleted
I20260807 05:12:33.173103 24942 catalog_manager.cc:4841] TS 18fd5bc0291745f7874ea50a73a243b3 (127.8.16.67:38607): tablet 97e1209932ce474ea21e384e366c8322 (table test-table [id=4f4ba4e020d14b2aa7ebb774182acf48]) successfully deleted
W20260807 05:12:33.174312 24944 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.16.65:38749 (error 108)
I20260807 05:12:33.174697 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 25174
I20260807 05:12:33.175710 25474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:33.175455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:38749 (local address 127.0.0.1:34274)
0807 05:12:33.175572 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:33.175638 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.65:38749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 05:12:33.175853 24944 catalog_manager.cc:4584] TS 15a96e9746cf467d93b56735f7572af5 (127.8.16.65:38749): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 97e1209932ce474ea21e384e366c8322: Network error: Client connection negotiation failed: client connection to 127.8.16.65:38749: connect: Connection refused (error 111)
I20260807 05:12:33.181056 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 25320
I20260807 05:12:33.187239 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 24921
I20260807 05:12:53.193714 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:41973
--webserver_interface=127.8.16.126
--webserver_port=33345
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:41973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:12:53.303153 25502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:12:53.303423 25502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:12:53.303480 25502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:12:53.307410 25502 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 05:12:53.307466 25502 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 05:12:53.307487 25502 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 05:12:53.307503 25502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:12:53.307525 25502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:12:53.307550 25502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:12:53.307579 25502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:12:53.307606 25502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:12:53.312824 25502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:41973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:41973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=33345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:12:53.313858 25502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:12:53.314839 25502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:12:53.322768 25507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:12:53.322788 25508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:12:53.322932 25502 server_base.cc:1034] running on GCE node
W20260807 05:12:53.322784 25510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:12:53.323606 25502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:12:53.324637 25502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:12:53.325802 25502 hybrid_clock.cc:648] HybridClock initialized: now 1786079573325787 us; error 31 us; skew 500 ppm
Aug 07 05:12:53 dist-test-slave-49h3 krb5kdc[24328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079573, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:12:53.328467 25502 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:12:53.329294 25502 webserver.cc:460] Webserver started at http://127.8.16.126:33345/ using document root <none> and password file <none>
I20260807 05:12:53.329834 25502 fs_manager.cc:359] Metadata directory not provided
I20260807 05:12:53.329901 25502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:12:53.333549 25502 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 05:12:53.335827 25517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:53.336993 25502 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:12:53.337098 25502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "d412a46cca7143a7a1aad6a57dc4ba18"
format_stamp: "Formatted at 2026-08-07 05:12:11 on dist-test-slave-49h3"
server_key: "0dec8bb2e91febe04c8884d03cdf8989"
server_key_iv: "6795e39dce447ddf59fdbb7d66e260fa"
server_key_version: "encryptionkey@0"
I20260807 05:12:53.337697 25502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:12:53.347378 25502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:12:53.350178 25502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:12:53.350342 25502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:12:53.356818 25502 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:41973
I20260807 05:12:53.357057 25577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:41973 every 8 connection(s)
I20260807 05:12:53.357859 25502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:12:53.360661 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 25502
I20260807 05:12:53.361063 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:38749
--local_ip_for_outbound_sockets=127.8.16.65
--tserver_master_addrs=127.8.16.126:41973
--webserver_port=33597
--webserver_interface=127.8.16.65
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:12:53.362154 25578 sys_catalog.cc:263] Verifying existing consensus state
I20260807 05:12:53.364207 25578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18: Bootstrap starting.
I20260807 05:12:53.381289 25578 log.cc:824] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18: Log is configured to *not* fsync() on all Append() calls
I20260807 05:12:53.392307 25578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:12:53.392709 25578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18: Bootstrap complete.
I20260807 05:12:53.396695 25578 raft_consensus.cc:348] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41973 } }
I20260807 05:12:53.397104 25578 raft_consensus.cc:729] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d412a46cca7143a7a1aad6a57dc4ba18, State: Initialized, Role: FOLLOWER
I20260807 05:12:53.397626 25578 consensus_queue.cc:260] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41973 } }
I20260807 05:12:53.397763 25578 raft_consensus.cc:388] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:12:53.397835 25578 raft_consensus.cc:482] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:12:53.397931 25578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 2 FOLLOWER]: Advancing to term 3
I20260807 05:12:53.399788 25578 raft_consensus.cc:504] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41973 } }
I20260807 05:12:53.400103 25578 leader_election.cc:302] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d412a46cca7143a7a1aad6a57dc4ba18; no voters:
I20260807 05:12:53.400404 25578 leader_election.cc:288] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 05:12:53.400506 25583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 05:12:53.400715 25583 raft_consensus.cc:686] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [term 3 LEADER]: Becoming Leader. State: Replica: d412a46cca7143a7a1aad6a57dc4ba18, State: Running, Role: LEADER
I20260807 05:12:53.401062 25583 consensus_queue.cc:237] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41973 } }
I20260807 05:12:53.401666 25578 sys_catalog.cc:564] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:12:53.402391 25584 sys_catalog.cc:455] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41973 } } }
I20260807 05:12:53.402515 25584 sys_catalog.cc:458] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [sys.catalog]: This master's current role is: LEADER
I20260807 05:12:53.403206 25585 sys_catalog.cc:455] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d412a46cca7143a7a1aad6a57dc4ba18. Latest consensus state: current_term: 3 leader_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d412a46cca7143a7a1aad6a57dc4ba18" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41973 } } }
I20260807 05:12:53.403335 25585 sys_catalog.cc:458] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18 [sys.catalog]: This master's current role is: LEADER
I20260807 05:12:53.403414 25591 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:12:53.407661 25591 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:12:53.408172 25591 catalog_manager.cc:1259] Loaded cluster ID: 3985e8fca84d40a7a434a40437e08ee3
I20260807 05:12:53.408223 25591 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:12:53.410035 25591 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:12:53.410615 25591 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18: Loaded TSK: 0
I20260807 05:12:53.410681 25591 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d412a46cca7143a7a1aad6a57dc4ba18: Loaded TSK: 1
F20260807 05:12:53.411228 25591 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1195faf9ad google::LogMessage::Fail() at ??:0
@ 0x7f1195fb288a google::LogMessage::SendToLog() at ??:0
@ 0x7f1195faf4de google::LogMessage::Flush() at ??:0
@ 0x7f1195fb0a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f1198c7318d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f1198c6bf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f11a09147bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f11a08ee99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f11a08ef178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f11a0930df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f11a094f82b std::function<>::operator()() at ??:0
@ 0x7f11a08eeb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f11a08eff3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f11a08ec579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f11a0930640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f11982013d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1198201c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f11982033f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f11981f2512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f11998366db start_thread at ??:0
@ 0x7f1194aad61f clone at ??:0
W20260807 05:12:53.502710 25580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:12:53.503015 25580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:12:53.503095 25580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:12:53.507149 25580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:12:53.507201 25580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:12:53.507272 25580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:12:53.512473 25580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:38749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=33597
--tserver_master_addrs=127.8.16.126:41973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:12:53.513510 25580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:12:53.514492 25580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:12:53.523566 25634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:12:53.523795 25580 server_base.cc:1034] running on GCE node
W20260807 05:12:53.523581 25632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:12:53.523693 25636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:12:53.524420 25580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:12:53.525223 25580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:12:53.526422 25580 hybrid_clock.cc:648] HybridClock initialized: now 1786079573526383 us; error 50 us; skew 500 ppm
Aug 07 05:12:53 dist-test-slave-49h3 krb5kdc[24328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079573, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:12:53.529744 25580 init.cc:377] Logged in from keytab as kudu/127.8.16.65@KRBTEST.COM (short username kudu)
I20260807 05:12:53.530596 25580 webserver.cc:460] Webserver started at http://127.8.16.65:33597/ using document root <none> and password file <none>
I20260807 05:12:53.531121 25580 fs_manager.cc:359] Metadata directory not provided
I20260807 05:12:53.531179 25580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:12:53.535035 25580 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:12:53.537130 25644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:53.538432 25580 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:12:53.538538 25580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "15a96e9746cf467d93b56735f7572af5"
format_stamp: "Formatted at 2026-08-07 05:12:11 on dist-test-slave-49h3"
server_key: "eb753fe2a4295f32ddaee063cf31770b"
server_key_iv: "91435cd66ef56cb135cf478cb19d5ea2"
server_key_version: "encryptionkey@0"
I20260807 05:12:53.539055 25580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:12:53.561400 25580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:12:53.564426 25580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:12:53.564603 25580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:12:53.565171 25580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:12:53.566291 25651 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 05:12:53.571517 25580 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 05:12:53.571575 25580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260807 05:12:53.571643 25580 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 05:12:53.573274 25580 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 05:12:53.573320 25580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260807 05:12:53.573664 25651 tablet_bootstrap.cc:492] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5: Bootstrap starting.
I20260807 05:12:53.582921 25580 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:38749
I20260807 05:12:53.583139 25774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:38749 every 8 connection(s)
I20260807 05:12:53.583863 25580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:12:53.588640 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 25580
I20260807 05:12:53.589046 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:46191
--local_ip_for_outbound_sockets=127.8.16.66
--tserver_master_addrs=127.8.16.126:41973
--webserver_port=36861
--webserver_interface=127.8.16.66
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:12:53.595479 25651 log.cc:824] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5: Log is configured to *not* fsync() on all Append() calls
I20260807 05:12:53.604185 25651 tablet_bootstrap.cc:492] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:12:53.604581 25651 tablet_bootstrap.cc:492] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5: Bootstrap complete.
I20260807 05:12:53.605821 25651 ts_tablet_manager.cc:1397] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5: Time spent bootstrapping tablet: real 0.032s user 0.013s sys 0.018s
I20260807 05:12:53.609009 25651 raft_consensus.cc:348] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } }
I20260807 05:12:53.609577 25651 raft_consensus.cc:729] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15a96e9746cf467d93b56735f7572af5, State: Initialized, Role: FOLLOWER
I20260807 05:12:53.610131 25651 consensus_queue.cc:260] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } }
I20260807 05:12:53.611204 25651 ts_tablet_manager.cc:1428] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
W20260807 05:12:53.626526 25777 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:12:53.585636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.65:58163)
0807 05:12:53.586114 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:53.586166 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 05:12:53.586944 (+ 778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:12:53.625584 (+ 38640us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":119,"threads_started":1}
W20260807 05:12:53.627035 25775 heartbeater.cc:644] Failed to heartbeat to 127.8.16.126:41973 (0 consecutive failures): Network error: Failed to ping master at 127.8.16.126:41973: Client connection negotiation failed: client connection to 127.8.16.126:41973: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 05:12:53.627755 25777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:53.627492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.65:51411)
0807 05:12:53.627623 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:53.627693 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 05:12:53.628334 25777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:53.628118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.65:37483)
0807 05:12:53.628224 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:53.628280 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 05:12:53.628481 25775 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 05:12:53.698896 25778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:12:53.699169 25778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:12:53.699222 25778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:12:53.703327 25778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:12:53.703394 25778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:12:53.703464 25778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:12:53.708823 25778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:46191
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=36861
--tserver_master_addrs=127.8.16.126:41973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:12:53.709882 25778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:12:53.710908 25778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:12:53.719770 25787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:12:53.719771 25790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:12:53.719971 25778 server_base.cc:1034] running on GCE node
W20260807 05:12:53.720310 25788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:12:53.720695 25778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:12:53.721261 25778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:12:53.722419 25778 hybrid_clock.cc:648] HybridClock initialized: now 1786079573722403 us; error 50 us; skew 500 ppm
Aug 07 05:12:53 dist-test-slave-49h3 krb5kdc[24328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079573, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:12:53.724901 25778 init.cc:377] Logged in from keytab as kudu/127.8.16.66@KRBTEST.COM (short username kudu)
I20260807 05:12:53.725745 25778 webserver.cc:460] Webserver started at http://127.8.16.66:36861/ using document root <none> and password file <none>
I20260807 05:12:53.726279 25778 fs_manager.cc:359] Metadata directory not provided
I20260807 05:12:53.726344 25778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:12:53.729918 25778 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:12:53.731750 25797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:53.732877 25778 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:12:53.732980 25778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "0d4b47e627f24cef9e22a5c6bf04bb05"
format_stamp: "Formatted at 2026-08-07 05:12:11 on dist-test-slave-49h3"
server_key: "c6f0933fe8a13eb007d0521140c1253b"
server_key_iv: "0d57ecc52845f1f05c8c2350331cc3ec"
server_key_version: "encryptionkey@0"
I20260807 05:12:53.733547 25778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:12:53.753962 25778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:12:53.756861 25778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:12:53.757050 25778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:12:53.757696 25778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:12:53.758528 25778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:12:53.758579 25778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:53.758632 25778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:12:53.758657 25778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:53.767372 25778 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:46191
I20260807 05:12:53.767401 25920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:46191 every 8 connection(s)
I20260807 05:12:53.768343 25778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:12:53.771873 25923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:53.770431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.66:55205)
0807 05:12:53.771070 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:53.771177 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":123,"threads_started":1}
W20260807 05:12:53.772302 25921 heartbeater.cc:644] Failed to heartbeat to 127.8.16.126:41973 (0 consecutive failures): Network error: Failed to ping master at 127.8.16.126:41973: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
I20260807 05:12:53.772879 25923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:53.772668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.66:50927)
0807 05:12:53.772777 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:53.772826 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 05:12:53.773427 25923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:53.773234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.66:47853)
0807 05:12:53.773335 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:53.773380 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 05:12:53.773625 25921 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 05:12:53.775550 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 25778
I20260807 05:12:53.775929 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:38607
--local_ip_for_outbound_sockets=127.8.16.67
--tserver_master_addrs=127.8.16.126:41973
--webserver_port=44625
--webserver_interface=127.8.16.67
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:12:53.883563 25924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:12:53.883837 25924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:12:53.883879 25924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:12:53.887814 25924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:12:53.887874 25924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:12:53.887929 25924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:12:53.892973 25924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:38607
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=44625
--tserver_master_addrs=127.8.16.126:41973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:12:53.893973 25924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:12:53.894909 25924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:12:53.903818 25930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:12:53.903820 25935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:12:53.903854 25924 server_base.cc:1034] running on GCE node
W20260807 05:12:53.903820 25932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:12:53.904489 25924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:12:53.905073 25924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:12:53.906277 25924 hybrid_clock.cc:648] HybridClock initialized: now 1786079573906257 us; error 35 us; skew 500 ppm
Aug 07 05:12:53 dist-test-slave-49h3 krb5kdc[24328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079573, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:12:53.908824 25924 init.cc:377] Logged in from keytab as kudu/127.8.16.67@KRBTEST.COM (short username kudu)
I20260807 05:12:53.909636 25924 webserver.cc:460] Webserver started at http://127.8.16.67:44625/ using document root <none> and password file <none>
I20260807 05:12:53.910137 25924 fs_manager.cc:359] Metadata directory not provided
I20260807 05:12:53.910200 25924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:12:53.913738 25924 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 05:12:53.915613 25942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:53.916836 25924 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:12:53.916937 25924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "18fd5bc0291745f7874ea50a73a243b3"
format_stamp: "Formatted at 2026-08-07 05:12:11 on dist-test-slave-49h3"
server_key: "80248c25c3a6cc7cd49e8699e9e11837"
server_key_iv: "710a7dbc29458e275700e26f5a3a88bb"
server_key_version: "encryptionkey@0"
I20260807 05:12:53.917440 25924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:12:53.943214 25924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:12:53.946327 25924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:12:53.946491 25924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:12:53.947038 25924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:12:53.947849 25924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:12:53.947897 25924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:53.947950 25924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:12:53.947976 25924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:12:53.956629 25924 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:38607
I20260807 05:12:53.957267 26077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:38607 every 8 connection(s)
I20260807 05:12:53.957887 25924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:12:53.961540 26080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:53.960021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.67:42111)
0807 05:12:53.960691 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:53.960813 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":113,"threads_started":1}
W20260807 05:12:53.961930 26078 heartbeater.cc:644] Failed to heartbeat to 127.8.16.126:41973 (0 consecutive failures): Network error: Failed to ping master at 127.8.16.126:41973: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
I20260807 05:12:53.962018 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 25924
I20260807 05:12:53.962523 26080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:53.962293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.67:50507)
0807 05:12:53.962401 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:53.962458 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 05:12:53.963105 26080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:53.962918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.67:43113)
0807 05:12:53.963009 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:53.963051 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 05:12:53.963299 26078 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 05:12:54.629413 26082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:54.629024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.65:56371)
0807 05:12:54.629248 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:54.629302 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":73,"threads_started":1}
I20260807 05:12:54.774576 26083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:54.774128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.66:38779)
0807 05:12:54.774414 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:54.774474 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":95,"threads_started":1}
I20260807 05:12:54.964154 26084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:54.963761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.67:57673)
0807 05:12:54.963971 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:54.964032 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":66,"threads_started":1}
I20260807 05:12:55.307988 26085 raft_consensus.cc:482] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:12:55.308178 26085 raft_consensus.cc:504] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } }
I20260807 05:12:55.309275 26085 leader_election.cc:288] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d4b47e627f24cef9e22a5c6bf04bb05 (127.8.16.66:46191), 18fd5bc0291745f7874ea50a73a243b3 (127.8.16.67:38607)
Aug 07 05:12:55 dist-test-slave-49h3 krb5kdc[24328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079573, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.66@KRBTEST.COM
Aug 07 05:12:55 dist-test-slave-49h3 krb5kdc[24328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079573, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.67@KRBTEST.COM
I20260807 05:12:55.322429 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:55.309700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:40961 (local address 127.8.16.66:46191)
0807 05:12:55.310441 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:55.310454 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:55.310576 (+ 122us) server_negotiation.cc:407] Connection header received
0807 05:12:55.311208 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:55.311213 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:55.311512 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:55.311780 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:12:55.312992 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:55.313983 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:55.315139 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:55.315577 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:55.315745 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:55.318213 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:12:55.318219 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:12:55.318235 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:12:55.318267 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:12:55.319990 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:55.320505 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:55.320512 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:55.320517 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:55.320629 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:55.320968 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:55.320971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:55.320972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:55.321369 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:12:55.321508 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:55.322006 (+ 498us) server_negotiation.cc:299] Negotiation successful
0807 05:12:55.322254 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":138,"threads_started":1}
I20260807 05:12:55.322443 26086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:55.309539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:46191 (local address 127.8.16.65:40961)
0807 05:12:55.309832 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:55.309848 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:12:55.309955 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:12:55.311854 (+ 1899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:12:55.311860 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:12:55.312369 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:12:55.312837 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:55.312852 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:55.314115 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:55.314118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:55.315026 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:55.315036 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:55.315698 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:55.315701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:55.315920 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:55.316496 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:12:55.316511 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:12:55.317984 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:12:55.320126 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:12:55.320129 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:12:55.320130 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:12:55.320403 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:12:55.320719 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:12:55.320722 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:12:55.320724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:12:55.320846 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:12:55.321496 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:12:55.321502 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:12:55.321729 (+ 227us) client_negotiation.cc:769] Sending connection context
0807 05:12:55.321883 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 05:12:55.322265 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":223,"spinlock_wait_cycles":13312,"thread_start_us":69,"threads_started":1}
I20260807 05:12:55.322453 26087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:55.309705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:42447 (local address 127.8.16.67:38607)
0807 05:12:55.310197 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0807 05:12:55.310219 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 05:12:55.310443 (+ 224us) server_negotiation.cc:407] Connection header received
0807 05:12:55.311175 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:12:55.311181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:12:55.311510 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:12:55.311787 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:12:55.312991 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:55.313988 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:55.315141 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:12:55.315575 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:12:55.315706 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:55.318516 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:12:55.318521 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:12:55.318532 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:12:55.318558 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:12:55.319969 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:55.320517 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:55.320522 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:55.320528 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:55.320658 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:12:55.320968 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:12:55.320971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:12:55.320973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:12:55.321484 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:12:55.321601 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 05:12:55.322004 (+ 403us) server_negotiation.cc:299] Negotiation successful
0807 05:12:55.322270 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":67,"threads_started":1}
I20260807 05:12:55.322877 26089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:55.309542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:38607 (local address 127.8.16.65:42447)
0807 05:12:55.310319 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:55.310334 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:12:55.310450 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:12:55.311856 (+ 1406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:12:55.311862 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:12:55.312369 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:12:55.312838 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:55.312852 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:55.314112 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:55.314114 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:55.315025 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:12:55.315034 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:12:55.315687 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:12:55.315690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:12:55.315922 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:12:55.316543 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:12:55.316554 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:12:55.318341 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:12:55.320077 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:12:55.320083 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:12:55.320096 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:12:55.320415 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:12:55.320763 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:12:55.320774 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:12:55.320776 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:12:55.320866 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:12:55.321585 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:12:55.321588 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:12:55.321726 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 05:12:55.321882 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 05:12:55.322764 (+ 882us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"mutex_wait_us":633,"spinlock_wait_cycles":22656,"thread_start_us":93,"threads_started":1}
W20260807 05:12:55.324743 25647 leader_election.cc:341] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d4b47e627f24cef9e22a5c6bf04bb05 (127.8.16.66:46191): Not found: Tablet not found: 97e1209932ce474ea21e384e366c8322
W20260807 05:12:55.325337 25646 leader_election.cc:341] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18fd5bc0291745f7874ea50a73a243b3 (127.8.16.67:38607): Not found: Tablet not found: 97e1209932ce474ea21e384e366c8322
I20260807 05:12:55.325409 25646 leader_election.cc:302] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 15a96e9746cf467d93b56735f7572af5; no voters: 0d4b47e627f24cef9e22a5c6bf04bb05, 18fd5bc0291745f7874ea50a73a243b3
I20260807 05:12:55.325649 26085 raft_consensus.cc:2726] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:12:55.630434 26089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:55.630066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.65:33879)
0807 05:12:55.630242 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:55.630326 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 05:12:55.775745 26090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:55.775294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.66:52955)
0807 05:12:55.775569 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:55.775630 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":97,"threads_started":1}
I20260807 05:12:55.965262 26091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:55.964858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.67:56553)
0807 05:12:55.965102 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:55.965157 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":89,"threads_started":1}
I20260807 05:12:56.631589 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:56.631187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.65:51199)
0807 05:12:56.631441 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:56.631491 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":95,"threads_started":1}
I20260807 05:12:56.776929 26093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:56.776472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.66:33213)
0807 05:12:56.776777 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:56.776835 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":105,"threads_started":1}
I20260807 05:12:56.966571 26094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:56.966037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.67:50523)
0807 05:12:56.966376 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:56.966445 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":110,"threads_started":1}
I20260807 05:12:57.123425 26095 raft_consensus.cc:482] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:12:57.123549 26095 raft_consensus.cc:504] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } }
I20260807 05:12:57.123917 26095 leader_election.cc:288] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d4b47e627f24cef9e22a5c6bf04bb05 (127.8.16.66:46191), 18fd5bc0291745f7874ea50a73a243b3 (127.8.16.67:38607)
W20260807 05:12:57.124742 25646 leader_election.cc:341] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18fd5bc0291745f7874ea50a73a243b3 (127.8.16.67:38607): Not found: Tablet not found: 97e1209932ce474ea21e384e366c8322
W20260807 05:12:57.124852 25647 leader_election.cc:341] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d4b47e627f24cef9e22a5c6bf04bb05 (127.8.16.66:46191): Not found: Tablet not found: 97e1209932ce474ea21e384e366c8322
I20260807 05:12:57.124935 25647 leader_election.cc:302] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 15a96e9746cf467d93b56735f7572af5; no voters: 0d4b47e627f24cef9e22a5c6bf04bb05, 18fd5bc0291745f7874ea50a73a243b3
I20260807 05:12:57.125066 26095 raft_consensus.cc:2726] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:12:57.632863 26096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:57.632415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.65:46747)
0807 05:12:57.632701 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:57.632766 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":83,"threads_started":1}
I20260807 05:12:57.778213 26097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:57.777751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.66:50179)
0807 05:12:57.778041 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:57.778115 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":80,"threads_started":1}
I20260807 05:12:57.968719 26098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:57.968222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.67:42357)
0807 05:12:57.968529 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:57.968590 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":117,"threads_started":1}
I20260807 05:12:58.634232 26099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:58.633759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.65:59589)
0807 05:12:58.634073 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:58.634136 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":78,"threads_started":1}
I20260807 05:12:58.779482 26100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:58.778998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.66:55041)
0807 05:12:58.779311 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:58.779379 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":102,"threads_started":1}
I20260807 05:12:58.969941 26101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:58.969541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.67:53055)
0807 05:12:58.969783 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:58.969842 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":68,"threads_started":1}
I20260807 05:12:59.635466 26102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:59.635019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.65:47843)
0807 05:12:59.635300 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:59.635363 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":86,"threads_started":1}
I20260807 05:12:59.780545 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:59.780142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.66:43181)
0807 05:12:59.780399 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:59.780453 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":74,"threads_started":1}
I20260807 05:12:59.971200 26104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:12:59.970692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.67:39705)
0807 05:12:59.971007 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 05:12:59.971075 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":110,"threads_started":1}
I20260807 05:13:00.032414 26105 raft_consensus.cc:482] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:13:00.032562 26105 raft_consensus.cc:504] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } }
I20260807 05:13:00.032968 26105 leader_election.cc:288] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d4b47e627f24cef9e22a5c6bf04bb05 (127.8.16.66:46191), 18fd5bc0291745f7874ea50a73a243b3 (127.8.16.67:38607)
W20260807 05:13:00.033815 25646 leader_election.cc:341] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18fd5bc0291745f7874ea50a73a243b3 (127.8.16.67:38607): Not found: Tablet not found: 97e1209932ce474ea21e384e366c8322
W20260807 05:13:00.033917 25647 leader_election.cc:341] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d4b47e627f24cef9e22a5c6bf04bb05 (127.8.16.66:46191): Not found: Tablet not found: 97e1209932ce474ea21e384e366c8322
I20260807 05:13:00.034016 25647 leader_election.cc:302] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 15a96e9746cf467d93b56735f7572af5; no voters: 0d4b47e627f24cef9e22a5c6bf04bb05, 18fd5bc0291745f7874ea50a73a243b3
I20260807 05:13:00.034171 26105 raft_consensus.cc:2726] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:13:00.636759 26106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:00.636301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.65:33589)
0807 05:13:00.636599 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:00.636662 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":94,"threads_started":1}
I20260807 05:13:00.781785 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:00.781245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.66:34367)
0807 05:13:00.781628 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:00.781688 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":75,"threads_started":1}
I20260807 05:13:00.972424 26108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:00.971966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.67:34581)
0807 05:13:00.972260 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:00.972327 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":79,"threads_started":1}
I20260807 05:13:01.637952 26109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:01.637500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.65:33837)
0807 05:13:01.637762 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:01.637829 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":75,"threads_started":1}
I20260807 05:13:01.782837 26110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:01.782422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.66:50555)
0807 05:13:01.782676 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:01.782729 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":92,"threads_started":1}
I20260807 05:13:01.973821 26111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:01.973284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.67:36509)
0807 05:13:01.973640 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:01.973717 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":90,"threads_started":1}
I20260807 05:13:02.639186 26112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:02.638679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.65:55479)
0807 05:13:02.639000 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:02.639069 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":101,"threads_started":1}
I20260807 05:13:02.783946 26113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:02.783523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.66:60783)
0807 05:13:02.783793 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:02.783850 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":80,"threads_started":1}
I20260807 05:13:02.975128 26114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:02.974654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.67:59933)
0807 05:13:02.974948 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:02.975017 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":80,"threads_started":1}
I20260807 05:13:03.640664 26115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:03.640147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.65:51423)
0807 05:13:03.640478 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:03.640553 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":122,"threads_started":1}
I20260807 05:13:03.785254 26116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:03.784750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.66:42141)
0807 05:13:03.785078 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:03.785149 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":100,"threads_started":1}
I20260807 05:13:03.976358 26117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:03.975901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.67:36347)
0807 05:13:03.976199 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:03.976260 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":82,"threads_started":1}
I20260807 05:13:04.641906 26118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:04.641414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.65:55473)
0807 05:13:04.641749 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:04.641804 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":101,"threads_started":1}
I20260807 05:13:04.786512 26119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:04.786041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.66:42439)
0807 05:13:04.786351 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:04.786412 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":100,"threads_started":1}
I20260807 05:13:04.977716 26120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:04.977146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.67:35289)
0807 05:13:04.977514 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:04.977608 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":88,"threads_started":1}
I20260807 05:13:04.999323 26121 raft_consensus.cc:482] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:13:04.999455 26121 raft_consensus.cc:504] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4b47e627f24cef9e22a5c6bf04bb05" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46191 } } peers { permanent_uuid: "18fd5bc0291745f7874ea50a73a243b3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38607 } } peers { permanent_uuid: "15a96e9746cf467d93b56735f7572af5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 38749 } }
I20260807 05:13:04.999882 26121 leader_election.cc:288] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d4b47e627f24cef9e22a5c6bf04bb05 (127.8.16.66:46191), 18fd5bc0291745f7874ea50a73a243b3 (127.8.16.67:38607)
W20260807 05:13:05.000612 25647 leader_election.cc:341] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d4b47e627f24cef9e22a5c6bf04bb05 (127.8.16.66:46191): Not found: Tablet not found: 97e1209932ce474ea21e384e366c8322
W20260807 05:13:05.000731 25646 leader_election.cc:341] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18fd5bc0291745f7874ea50a73a243b3 (127.8.16.67:38607): Not found: Tablet not found: 97e1209932ce474ea21e384e366c8322
I20260807 05:13:05.000828 25646 leader_election.cc:302] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 15a96e9746cf467d93b56735f7572af5; no voters: 0d4b47e627f24cef9e22a5c6bf04bb05, 18fd5bc0291745f7874ea50a73a243b3
I20260807 05:13:05.000972 26121 raft_consensus.cc:2726] T 97e1209932ce474ea21e384e366c8322 P 15a96e9746cf467d93b56735f7572af5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:13:05.643052 26122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:05.642617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.65:52333)
0807 05:13:05.642887 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:05.642952 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":90,"threads_started":1}
I20260807 05:13:05.787804 26123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:05.787339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.66:38709)
0807 05:13:05.787624 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:05.787690 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":80,"threads_started":1}
I20260807 05:13:05.978965 26125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:05.978491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.67:41017)
0807 05:13:05.978785 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:05.978847 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":101,"threads_started":1}
I20260807 05:13:06.644264 26126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:06.643816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.65:38723)
0807 05:13:06.644088 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:06.644160 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":80,"threads_started":1}
I20260807 05:13:06.789245 26127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:06.788740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.66:45437)
0807 05:13:06.789061 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:06.789128 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":85,"threads_started":1}
I20260807 05:13:06.980275 26128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:06.979779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.67:56425)
0807 05:13:06.980110 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:06.980177 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":104,"threads_started":1}
I20260807 05:13:07.645328 26129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:07.644925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.65:44961)
0807 05:13:07.645171 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:07.645226 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":78,"threads_started":1}
I20260807 05:13:07.790513 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:07.790078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.66:45883)
0807 05:13:07.790344 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:07.790409 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":85,"threads_started":1}
I20260807 05:13:07.981467 26131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:07.981031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.67:37605)
0807 05:13:07.981288 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:07.981344 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":90,"threads_started":1}
I20260807 05:13:08.646724 26134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:08.646294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.65:55533)
0807 05:13:08.646563 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:08.646628 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":80,"threads_started":1}
I20260807 05:13:08.791936 26135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:08.791474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41973 (local address 127.8.16.66:39101)
0807 05:13:08.791776 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:08.791838 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.126:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":77,"threads_started":1}
I20260807 05:13:08.963580 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 25580
I20260807 05:13:08.970548 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 25778
I20260807 05:13:08.976603 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 25924
2026-08-07T05:13:08Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59120 ms)
[----------] 30 tests from SecurityITest (234900 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:13:09 dist-test-slave-49h3 krb5kdc[26138](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:13:09 dist-test-slave-49h3 krb5kdc[26138](info): set up 2 sockets
Aug 07 05:13:09 dist-test-slave-49h3 krb5kdc[26138](info): commencing operation
krb5kdc: starting...
W20260807 05:13:09.978814 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.980s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:13:10 dist-test-slave-49h3 krb5kdc[26138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:13:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:13:10.097215 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:36779
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:36779
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:10.206811 26152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:10.207069 26152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:10.207113 26152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:10.211061 26152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:10.211117 26152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:10.211134 26152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:10.211148 26152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:10.211164 26152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:10.216411 26152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:36779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:36779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:10.217473 26152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:10.218467 26152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 05:13:10.226758 26152 server_base.cc:1034] running on GCE node
W20260807 05:13:10.226765 26158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:10.226765 26161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:10.226770 26159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:10.227623 26152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:10.228683 26152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:10.229857 26152 hybrid_clock.cc:648] HybridClock initialized: now 1786079590229840 us; error 27 us; skew 500 ppm
Aug 07 05:13:10 dist-test-slave-49h3 krb5kdc[26138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079590, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:10.232537 26152 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:13:10.233492 26152 webserver.cc:460] Webserver started at http://127.8.16.126:35831/ using document root <none> and password file <none>
I20260807 05:13:10.234031 26152 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:10.234107 26152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:10.234308 26152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:10.236315 26152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "f8737003b18b482ab5bb1ffb6794c741"
format_stamp: "Formatted at 2026-08-07 05:13:10 on dist-test-slave-49h3"
server_key: "bf8e19640f3ac98afeafc92a687d0de6"
server_key_iv: "be33702a6e3746bff945010c69ee6e89"
server_key_version: "encryptionkey@0"
I20260807 05:13:10.236861 26152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "f8737003b18b482ab5bb1ffb6794c741"
format_stamp: "Formatted at 2026-08-07 05:13:10 on dist-test-slave-49h3"
server_key: "bf8e19640f3ac98afeafc92a687d0de6"
server_key_iv: "be33702a6e3746bff945010c69ee6e89"
server_key_version: "encryptionkey@0"
I20260807 05:13:10.240375 26152 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:13:10.242604 26169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:10.243803 26152 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:13:10.243923 26152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "f8737003b18b482ab5bb1ffb6794c741"
format_stamp: "Formatted at 2026-08-07 05:13:10 on dist-test-slave-49h3"
server_key: "bf8e19640f3ac98afeafc92a687d0de6"
server_key_iv: "be33702a6e3746bff945010c69ee6e89"
server_key_version: "encryptionkey@0"
I20260807 05:13:10.244019 26152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:10.252842 26152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:10.255378 26152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:10.255544 26152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:10.261986 26152 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:36779
I20260807 05:13:10.262001 26229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:36779 every 8 connection(s)
I20260807 05:13:10.262887 26152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:10.263474 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 26152
I20260807 05:13:10.263573 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:10.263846 8257 external_mini_cluster.cc:1428] Setting key 95a4334e2510e3a0d485e300425727cc
I20260807 05:13:10.265887 26231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:10.272327 26231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741: Bootstrap starting.
Aug 07 05:13:10 dist-test-slave-49h3 krb5kdc[26138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:13:10.274995 26231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:10.275831 26231 log.cc:824] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:10.277814 26231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741: No bootstrap required, opened a new log
I20260807 05:13:10.278280 26234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:10.265199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34654 (local address 127.8.16.126:36779)
0807 05:13:10.265650 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:10.265667 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:10.265709 (+ 42us) server_negotiation.cc:407] Connection header received
0807 05:13:10.266540 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:10.266562 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:10.266957 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:10.267322 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:10.267988 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:10.269525 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:10.270319 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:10.270982 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:10.271139 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:10.273649 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:10.273655 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:10.273671 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:10.273703 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:10.275522 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:10.275930 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:10.275936 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:10.275942 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:10.276076 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:10.276341 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:10.276345 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:10.276346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:10.276755 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:10.276938 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:10.277210 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 05:13:10.277539 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":118,"threads_started":1}
I20260807 05:13:10.281428 26231 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8737003b18b482ab5bb1ffb6794c741" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36779 } }
I20260807 05:13:10.281742 26231 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:10.281852 26231 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8737003b18b482ab5bb1ffb6794c741, State: Initialized, Role: FOLLOWER
I20260807 05:13:10.282298 26231 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8737003b18b482ab5bb1ffb6794c741" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36779 } }
I20260807 05:13:10.282428 26231 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:10.282490 26231 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:10.282579 26231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:10.283445 26231 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8737003b18b482ab5bb1ffb6794c741" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36779 } }
I20260807 05:13:10.283758 26231 leader_election.cc:302] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8737003b18b482ab5bb1ffb6794c741; no voters:
I20260807 05:13:10.284051 26231 leader_election.cc:288] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:10.284168 26236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:10.284390 26236 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741 [term 1 LEADER]: Becoming Leader. State: Replica: f8737003b18b482ab5bb1ffb6794c741, State: Running, Role: LEADER
I20260807 05:13:10.284727 26236 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8737003b18b482ab5bb1ffb6794c741" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36779 } }
I20260807 05:13:10.285185 26231 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:10.286402 26237 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8737003b18b482ab5bb1ffb6794c741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8737003b18b482ab5bb1ffb6794c741" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36779 } } }
I20260807 05:13:10.286546 26237 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:10.286401 26238 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8737003b18b482ab5bb1ffb6794c741. Latest consensus state: current_term: 1 leader_uuid: "f8737003b18b482ab5bb1ffb6794c741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8737003b18b482ab5bb1ffb6794c741" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36779 } } }
I20260807 05:13:10.286762 26238 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:10.287063 26245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:10.290304 26245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:10.295449 26245 catalog_manager.cc:1347] Generated new cluster ID: 1ef09f3f1093451db3cd6bee06dfe26e
I20260807 05:13:10.295508 26245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:10.313007 26245 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:10.314052 26245 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:10.321094 26245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741: Generated new TSK 0
I20260807 05:13:10.321799 26245 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
I20260807 05:13:10.348315 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36779
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:10.468981 26258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:10.469249 26258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:10.469300 26258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:10.473383 26258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:10.473482 26258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:10.473582 26258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:13:10.478948 26258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:10.480059 26258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:10.481026 26258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:10.490387 26268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:10.490381 26265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:10.490485 26258 server_base.cc:1034] running on GCE node
W20260807 05:13:10.490381 26266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:10.491046 26258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:10.491631 26258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:10.492789 26258 hybrid_clock.cc:648] HybridClock initialized: now 1786079590492773 us; error 31 us; skew 500 ppm
Aug 07 05:13:10 dist-test-slave-49h3 krb5kdc[26138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079590, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:10.495362 26258 init.cc:377] Logged in from keytab as kudu/127.8.16.65@KRBTEST.COM (short username kudu)
I20260807 05:13:10.496222 26258 webserver.cc:460] Webserver started at http://127.8.16.65:43271/ using document root <none> and password file <none>
I20260807 05:13:10.496752 26258 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:10.496814 26258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:10.497014 26258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:10.499091 26258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "7112e76ac9574e27918a9e611846ce81"
format_stamp: "Formatted at 2026-08-07 05:13:10 on dist-test-slave-49h3"
server_key: "621bde3b54e9cf1722c7048d7a10aacb"
server_key_iv: "b41720c49a382788704148e322ce6490"
server_key_version: "encryptionkey@0"
I20260807 05:13:10.499624 26258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "7112e76ac9574e27918a9e611846ce81"
format_stamp: "Formatted at 2026-08-07 05:13:10 on dist-test-slave-49h3"
server_key: "621bde3b54e9cf1722c7048d7a10aacb"
server_key_iv: "b41720c49a382788704148e322ce6490"
server_key_version: "encryptionkey@0"
I20260807 05:13:10.503328 26258 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260807 05:13:10.505640 26275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:10.506795 26258 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:13:10.506911 26258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "7112e76ac9574e27918a9e611846ce81"
format_stamp: "Formatted at 2026-08-07 05:13:10 on dist-test-slave-49h3"
server_key: "621bde3b54e9cf1722c7048d7a10aacb"
server_key_iv: "b41720c49a382788704148e322ce6490"
server_key_version: "encryptionkey@0"
I20260807 05:13:10.507006 26258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:10.527021 26258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:10.529793 26258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:10.529969 26258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:10.530535 26258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:13:10.531322 26258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:13:10.531369 26258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:10.531422 26258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:13:10.531448 26258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:10.539690 26258 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:45881
I20260807 05:13:10.539732 26407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:45881 every 8 connection(s)
I20260807 05:13:10.540534 26258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:13:10.545192 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 26258
I20260807 05:13:10.545322 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:13:10.545630 8257 external_mini_cluster.cc:1428] Setting key 4831f4117ec3e53d08ed2ea7503a80e1
Aug 07 05:13:10 dist-test-slave-49h3 krb5kdc[26138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079590, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:13:10.553838 26234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:10.542264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:56319 (local address 127.8.16.126:36779)
0807 05:13:10.542394 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:10.542398 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:10.543178 (+ 780us) server_negotiation.cc:407] Connection header received
0807 05:13:10.544368 (+ 1190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:10.544370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:10.544415 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:10.544483 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:10.545665 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:10.546361 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:10.547690 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:10.548048 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:10.548143 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:10.550626 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:10.550629 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:10.550632 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:10.550654 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:10.551954 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:10.552424 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:10.552427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:10.552428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:10.552499 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:10.552809 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:10.552813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:10.552815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:10.552984 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:10.553068 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:10.553545 (+ 477us) server_negotiation.cc:299] Negotiation successful
0807 05:13:10.553673 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
WARNING: no policy specified for kudu/127.8.16.66@KRBTEST.COM; defaulting to no policy
I20260807 05:13:10.554646 26410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:10.542603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36779 (local address 127.8.16.65:56319)
0807 05:13:10.543027 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:10.543067 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:10.544177 (+ 1110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:10.544623 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:10.544629 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:10.544966 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:10.545520 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:10.545539 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:10.546483 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:10.546486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:10.547584 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:10.547594 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:10.548176 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:10.548179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:10.548327 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:10.548898 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:10.548919 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:10.550477 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:10.552062 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:10.552068 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:10.552083 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:10.552333 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:10.552580 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:10.552583 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:10.552585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:10.552725 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:10.553080 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:10.553087 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:10.553316 (+ 229us) client_negotiation.cc:769] Sending connection context
0807 05:13:10.553501 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 05:13:10.553825 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":109,"threads_started":1}
I20260807 05:13:10.555783 26408 heartbeater.cc:347] Connected to a master server at 127.8.16.126:36779
I20260807 05:13:10.556005 26408 heartbeater.cc:464] Registering TS with master...
I20260807 05:13:10.556468 26408 heartbeater.cc:511] Master 127.8.16.126:36779 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.66@KRBTEST.COM" created.
I20260807 05:13:10.557915 26186 ts_manager.cc:194] Registered new tserver with Master: 7112e76ac9574e27918a9e611846ce81 (127.8.16.65:45881)
I20260807 05:13:10.559006 26186 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:56319
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
I20260807 05:13:10.571678 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36779
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:10.680564 26413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:10.680830 26413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:10.680876 26413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:10.684885 26413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:10.684968 26413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:10.685048 26413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:13:10.690445 26413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:10.691466 26413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:10.692451 26413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:10.701493 26422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:10.701551 26420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:10.701655 26419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:10.701891 26413 server_base.cc:1034] running on GCE node
I20260807 05:13:10.702279 26413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:10.702886 26413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:10.704044 26413 hybrid_clock.cc:648] HybridClock initialized: now 1786079590704023 us; error 31 us; skew 500 ppm
Aug 07 05:13:10 dist-test-slave-49h3 krb5kdc[26138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079590, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:10.706620 26413 init.cc:377] Logged in from keytab as kudu/127.8.16.66@KRBTEST.COM (short username kudu)
I20260807 05:13:10.707439 26413 webserver.cc:460] Webserver started at http://127.8.16.66:36943/ using document root <none> and password file <none>
I20260807 05:13:10.707983 26413 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:10.708048 26413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:10.708231 26413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:10.710255 26413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "34e83edb4e374c81ad2834a54a7ec0b3"
format_stamp: "Formatted at 2026-08-07 05:13:10 on dist-test-slave-49h3"
server_key: "18393dd91684e9e88095e38f224f7152"
server_key_iv: "49d159e69c2f46b7bccbac87e13bb4ca"
server_key_version: "encryptionkey@0"
I20260807 05:13:10.710835 26413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "34e83edb4e374c81ad2834a54a7ec0b3"
format_stamp: "Formatted at 2026-08-07 05:13:10 on dist-test-slave-49h3"
server_key: "18393dd91684e9e88095e38f224f7152"
server_key_iv: "49d159e69c2f46b7bccbac87e13bb4ca"
server_key_version: "encryptionkey@0"
I20260807 05:13:10.714419 26413 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:13:10.716722 26429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:10.717893 26413 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:13:10.718001 26413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "34e83edb4e374c81ad2834a54a7ec0b3"
format_stamp: "Formatted at 2026-08-07 05:13:10 on dist-test-slave-49h3"
server_key: "18393dd91684e9e88095e38f224f7152"
server_key_iv: "49d159e69c2f46b7bccbac87e13bb4ca"
server_key_version: "encryptionkey@0"
I20260807 05:13:10.718093 26413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:10.737651 26413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:10.740326 26413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:10.740505 26413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:10.741087 26413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:13:10.741927 26413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:13:10.741987 26413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:10.742043 26413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:13:10.742069 26413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:10.750891 26413 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:37307
I20260807 05:13:10.750916 26556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:37307 every 8 connection(s)
I20260807 05:13:10.751750 26413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:13:10.758520 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 26413
I20260807 05:13:10.758646 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:13:10.758975 8257 external_mini_cluster.cc:1428] Setting key 321317f33caec3c2aabfc9a508655b78
Aug 07 05:13:10 dist-test-slave-49h3 krb5kdc[26138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079590, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:13:10.764858 26234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:10.753372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:37625 (local address 127.8.16.126:36779)
0807 05:13:10.753542 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:10.753547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:10.754391 (+ 844us) server_negotiation.cc:407] Connection header received
0807 05:13:10.755583 (+ 1192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:10.755586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:10.755633 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:10.755705 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:10.756811 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:10.757345 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:10.758553 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:10.758879 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:10.758987 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:10.761516 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:10.761520 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:10.761522 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:10.761545 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:10.762962 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:10.763418 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:10.763422 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:10.763424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:10.763514 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:10.763815 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:10.763818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:10.763820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:10.764000 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:10.764090 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:10.764563 (+ 473us) server_negotiation.cc:299] Negotiation successful
0807 05:13:10.764692 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 05:13:10.765772 26559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:10.753798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36779 (local address 127.8.16.66:37625)
0807 05:13:10.754243 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:10.754285 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:10.755398 (+ 1113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:10.755846 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:10.755852 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:10.756197 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:10.756676 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:10.756689 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:10.757497 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:10.757501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:10.758425 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:10.758437 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:10.759022 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:10.759026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:10.759191 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:10.759752 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:10.759768 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:10.761325 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:10.763070 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:10.763074 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:10.763086 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:10.763330 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:10.763585 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:10.763587 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:10.763589 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:10.763719 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:10.764101 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:10.764107 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:10.764340 (+ 233us) client_negotiation.cc:769] Sending connection context
0807 05:13:10.764502 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 05:13:10.764831 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":122,"threads_started":1}
I20260807 05:13:10.766906 26557 heartbeater.cc:347] Connected to a master server at 127.8.16.126:36779
I20260807 05:13:10.767143 26557 heartbeater.cc:464] Registering TS with master...
I20260807 05:13:10.767611 26557 heartbeater.cc:511] Master 127.8.16.126:36779 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.16.67@KRBTEST.COM; defaulting to no policy
I20260807 05:13:10.768620 26186 ts_manager.cc:194] Registered new tserver with Master: 34e83edb4e374c81ad2834a54a7ec0b3 (127.8.16.66:37307)
I20260807 05:13:10.769205 26186 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:37625
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
I20260807 05:13:10.784296 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36779
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:10.894289 26562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:10.894559 26562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:10.894603 26562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:10.898801 26562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:10.898861 26562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:10.898921 26562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:13:10.904181 26562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:10.905133 26562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:10.906162 26562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:10.915589 26571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:10.915589 26569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:10.915599 26568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:10.915809 26562 server_base.cc:1034] running on GCE node
I20260807 05:13:10.916293 26562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:10.916929 26562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:10.918104 26562 hybrid_clock.cc:648] HybridClock initialized: now 1786079590918087 us; error 27 us; skew 500 ppm
Aug 07 05:13:10 dist-test-slave-49h3 krb5kdc[26138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079590, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:10.920889 26562 init.cc:377] Logged in from keytab as kudu/127.8.16.67@KRBTEST.COM (short username kudu)
I20260807 05:13:10.921844 26562 webserver.cc:460] Webserver started at http://127.8.16.67:35565/ using document root <none> and password file <none>
I20260807 05:13:10.922374 26562 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:10.922439 26562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:10.922622 26562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:10.924633 26562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "bc74b2e646be42ccaf5f225c23b0872a"
format_stamp: "Formatted at 2026-08-07 05:13:10 on dist-test-slave-49h3"
server_key: "e08e0eaad579c54042eac2a1421cc1ff"
server_key_iv: "9195bd913b929dbbaa4b956be2b9640c"
server_key_version: "encryptionkey@0"
I20260807 05:13:10.925181 26562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "bc74b2e646be42ccaf5f225c23b0872a"
format_stamp: "Formatted at 2026-08-07 05:13:10 on dist-test-slave-49h3"
server_key: "e08e0eaad579c54042eac2a1421cc1ff"
server_key_iv: "9195bd913b929dbbaa4b956be2b9640c"
server_key_version: "encryptionkey@0"
I20260807 05:13:10.928763 26562 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:13:10.931078 26578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:10.932229 26562 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:13:10.932336 26562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "bc74b2e646be42ccaf5f225c23b0872a"
format_stamp: "Formatted at 2026-08-07 05:13:10 on dist-test-slave-49h3"
server_key: "e08e0eaad579c54042eac2a1421cc1ff"
server_key_iv: "9195bd913b929dbbaa4b956be2b9640c"
server_key_version: "encryptionkey@0"
I20260807 05:13:10.932426 26562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:10.947590 26562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:10.950359 26562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:10.950536 26562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:10.951128 26562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:13:10.951922 26562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:13:10.951969 26562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:10.952019 26562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:13:10.952047 26562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:10.960621 26562 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:38633
I20260807 05:13:10.960671 26704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:38633 every 8 connection(s)
I20260807 05:13:10.961555 26562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
Aug 07 05:13:10 dist-test-slave-49h3 krb5kdc[26138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079590, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:13:10.971418 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 26562
I20260807 05:13:10.971570 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:13:10.971930 8257 external_mini_cluster.cc:1428] Setting key caa42480ff53ef6a68c0e88b6836ebd5
I20260807 05:13:10.974689 26234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:10.963395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:48979 (local address 127.8.16.126:36779)
0807 05:13:10.963563 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:10.963568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:10.964399 (+ 831us) server_negotiation.cc:407] Connection header received
0807 05:13:10.965640 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:10.965643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:10.965692 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:10.965769 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:10.966940 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:10.967480 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:10.968566 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:10.968769 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:10.968834 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:10.971258 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:10.971262 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:10.971265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:10.971294 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:10.972668 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:10.973162 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:10.973165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:10.973166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:10.973252 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:10.973617 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:10.973621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:10.973623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:10.973807 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:10.973894 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:10.974398 (+ 504us) server_negotiation.cc:299] Negotiation successful
0807 05:13:10.974526 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 05:13:10.975626 26707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:10.963801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36779 (local address 127.8.16.67:48979)
0807 05:13:10.964246 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:10.964288 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:10.965419 (+ 1131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:10.965918 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:10.965926 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:10.966304 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:10.966788 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:10.966802 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:10.967574 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:10.967576 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:10.968473 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:10.968483 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:10.968876 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:10.968878 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:10.969013 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:10.969615 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:10.969632 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:10.971059 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:10.972787 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:10.972794 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:10.972811 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:10.973066 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:10.973338 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:10.973341 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:10.973343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:10.973531 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:10.973908 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:10.973915 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:10.974165 (+ 250us) client_negotiation.cc:769] Sending connection context
0807 05:13:10.974322 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 05:13:10.974649 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":128,"threads_started":1}
I20260807 05:13:10.976675 26705 heartbeater.cc:347] Connected to a master server at 127.8.16.126:36779
I20260807 05:13:10.976892 26705 heartbeater.cc:464] Registering TS with master...
I20260807 05:13:10.977326 26705 heartbeater.cc:511] Master 127.8.16.126:36779 requested a full tablet report, sending...
I20260807 05:13:10.978251 26186 ts_manager.cc:194] Registered new tserver with Master: bc74b2e646be42ccaf5f225c23b0872a (127.8.16.67:38633)
I20260807 05:13:10.978797 26186 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:48979
I20260807 05:13:10.985718 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:13:10.994506 26234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:10.987687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34658 (local address 127.8.16.126:36779)
0807 05:13:10.987823 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:10.987828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:10.987970 (+ 142us) server_negotiation.cc:407] Connection header received
0807 05:13:10.988093 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:10.988096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:10.988139 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:10.988203 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:10.988894 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:10.989411 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:10.990260 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:10.990475 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:10.990557 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:10.991837 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:10.991841 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:10.991844 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:10.991869 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:10.992947 (+ 1078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:10.993322 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:10.993325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:10.993327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:10.993405 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:10.993688 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:10.993691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:10.993693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:10.993909 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:10.994036 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:10.994251 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 05:13:10.994357 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 05:13:10.997110 26186 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:13:11.000075 26186 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:13:11.015823 26725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:11.010683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52384 (local address 127.8.16.65:45881)
0807 05:13:11.011257 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:11.011273 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:11.011295 (+ 22us) server_negotiation.cc:407] Connection header received
0807 05:13:11.011362 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:11.011367 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:11.011532 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:11.011665 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:11.012037 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.012932 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.014100 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.014898 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.015005 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:11.015163 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:11.015545 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 05:13:11.015646 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":111,"threads_started":1}
I20260807 05:13:11.015820 26722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:11.008891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60548 (local address 127.8.16.66:37307)
0807 05:13:11.009392 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:11.009404 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:11.009980 (+ 576us) server_negotiation.cc:407] Connection header received
0807 05:13:11.010109 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:11.010114 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:11.010978 (+ 864us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:11.011100 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:11.011316 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.012546 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.014492 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.015205 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.015299 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:11.015434 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:11.015594 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 05:13:11.015671 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":146,"threads_started":1}
I20260807 05:13:11.015949 26719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:11.008089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:38633 (local address 127.0.0.1:43586)
0807 05:13:11.009790 (+ 1701us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:11.009820 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:11.009967 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:11.010793 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:11.010799 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:11.010873 (+ 74us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:11.011009 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:11.011015 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.012673 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.012677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:11.014163 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:11.014174 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.015585 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.015588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:11.015685 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:11.015705 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 05:13:11.015764 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 05:13:11.015826 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1504,"mutex_wait_us":92,"spinlock_wait_cycles":6656,"thread_start_us":94,"threads_started":1}
I20260807 05:13:11.016328 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:11.008286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43586 (local address 127.8.16.67:38633)
0807 05:13:11.009063 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:11.009102 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:11.009922 (+ 820us) server_negotiation.cc:407] Connection header received
0807 05:13:11.010435 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:11.010440 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:11.010625 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:11.010799 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:11.011127 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.012537 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.014280 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.015039 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.015901 (+ 862us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:11.016035 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:11.016088 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 05:13:11.016178 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":157,"threads_started":1}
I20260807 05:13:11.017026 26721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:11.008357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:37307 (local address 127.0.0.1:60548)
0807 05:13:11.009766 (+ 1409us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:11.009820 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:11.009967 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:11.011099 (+ 1132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:11.011102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:11.011121 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:11.011224 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:11.011230 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.013199 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.013202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:11.014391 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:11.014398 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.015309 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.015313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:11.015402 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:11.015423 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 05:13:11.015502 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 05:13:11.016301 (+ 799us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1257,"mutex_wait_us":711,"spinlock_wait_cycles":10240,"thread_start_us":106,"threads_started":1}
I20260807 05:13:11.015805 26723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:11.010436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:45881 (local address 127.0.0.1:52384)
0807 05:13:11.010721 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:11.010745 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:11.010820 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:11.011831 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:11.011833 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:11.011847 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:11.011917 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:11.011921 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.013064 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.013067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:11.013978 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:11.013992 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.015220 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.015224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:11.015342 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:11.015382 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 05:13:11.015495 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 05:13:11.015645 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":30,"thread_start_us":97,"threads_started":1}
I20260807 05:13:11.019164 26632 tablet_service.cc:1467] Processing CreateTablet for tablet d1574c5c6822427a8b2921c1f113f5d1 (DEFAULT_TABLE table=test-table [id=1884a95964094705afca6bdfbd0a3f07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:13:11.019608 26485 tablet_service.cc:1467] Processing CreateTablet for tablet d1574c5c6822427a8b2921c1f113f5d1 (DEFAULT_TABLE table=test-table [id=1884a95964094705afca6bdfbd0a3f07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:13:11.019613 26331 tablet_service.cc:1467] Processing CreateTablet for tablet d1574c5c6822427a8b2921c1f113f5d1 (DEFAULT_TABLE table=test-table [id=1884a95964094705afca6bdfbd0a3f07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:13:11.020264 26632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1574c5c6822427a8b2921c1f113f5d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:11.020599 26485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1574c5c6822427a8b2921c1f113f5d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:11.020599 26331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1574c5c6822427a8b2921c1f113f5d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:11.026434 26726 tablet_bootstrap.cc:492] T d1574c5c6822427a8b2921c1f113f5d1 P bc74b2e646be42ccaf5f225c23b0872a: Bootstrap starting.
I20260807 05:13:11.027835 26728 tablet_bootstrap.cc:492] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81: Bootstrap starting.
I20260807 05:13:11.029220 26726 tablet_bootstrap.cc:654] T d1574c5c6822427a8b2921c1f113f5d1 P bc74b2e646be42ccaf5f225c23b0872a: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:11.029910 26728 tablet_bootstrap.cc:654] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:11.030320 26727 tablet_bootstrap.cc:492] T d1574c5c6822427a8b2921c1f113f5d1 P 34e83edb4e374c81ad2834a54a7ec0b3: Bootstrap starting.
I20260807 05:13:11.030556 26726 log.cc:824] T d1574c5c6822427a8b2921c1f113f5d1 P bc74b2e646be42ccaf5f225c23b0872a: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:11.032141 26728 log.cc:824] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:11.032369 26726 tablet_bootstrap.cc:492] T d1574c5c6822427a8b2921c1f113f5d1 P bc74b2e646be42ccaf5f225c23b0872a: No bootstrap required, opened a new log
I20260807 05:13:11.032553 26726 ts_tablet_manager.cc:1397] T d1574c5c6822427a8b2921c1f113f5d1 P bc74b2e646be42ccaf5f225c23b0872a: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260807 05:13:11.032723 26727 tablet_bootstrap.cc:654] T d1574c5c6822427a8b2921c1f113f5d1 P 34e83edb4e374c81ad2834a54a7ec0b3: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:11.033368 26727 log.cc:824] T d1574c5c6822427a8b2921c1f113f5d1 P 34e83edb4e374c81ad2834a54a7ec0b3: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:11.034132 26728 tablet_bootstrap.cc:492] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81: No bootstrap required, opened a new log
I20260807 05:13:11.034317 26728 ts_tablet_manager.cc:1397] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 05:13:11.034929 26727 tablet_bootstrap.cc:492] T d1574c5c6822427a8b2921c1f113f5d1 P 34e83edb4e374c81ad2834a54a7ec0b3: No bootstrap required, opened a new log
I20260807 05:13:11.035127 26727 ts_tablet_manager.cc:1397] T d1574c5c6822427a8b2921c1f113f5d1 P 34e83edb4e374c81ad2834a54a7ec0b3: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260807 05:13:11.036439 26726 raft_consensus.cc:348] T d1574c5c6822427a8b2921c1f113f5d1 P bc74b2e646be42ccaf5f225c23b0872a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc74b2e646be42ccaf5f225c23b0872a" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38633 } } peers { permanent_uuid: "34e83edb4e374c81ad2834a54a7ec0b3" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37307 } } peers { permanent_uuid: "7112e76ac9574e27918a9e611846ce81" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45881 } }
I20260807 05:13:11.036824 26726 raft_consensus.cc:374] T d1574c5c6822427a8b2921c1f113f5d1 P bc74b2e646be42ccaf5f225c23b0872a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:11.036923 26726 raft_consensus.cc:729] T d1574c5c6822427a8b2921c1f113f5d1 P bc74b2e646be42ccaf5f225c23b0872a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc74b2e646be42ccaf5f225c23b0872a, State: Initialized, Role: FOLLOWER
I20260807 05:13:11.037438 26726 consensus_queue.cc:260] T d1574c5c6822427a8b2921c1f113f5d1 P bc74b2e646be42ccaf5f225c23b0872a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc74b2e646be42ccaf5f225c23b0872a" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38633 } } peers { permanent_uuid: "34e83edb4e374c81ad2834a54a7ec0b3" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37307 } } peers { permanent_uuid: "7112e76ac9574e27918a9e611846ce81" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45881 } }
I20260807 05:13:11.037977 26728 raft_consensus.cc:348] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc74b2e646be42ccaf5f225c23b0872a" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38633 } } peers { permanent_uuid: "34e83edb4e374c81ad2834a54a7ec0b3" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37307 } } peers { permanent_uuid: "7112e76ac9574e27918a9e611846ce81" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45881 } }
I20260807 05:13:11.038151 26705 heartbeater.cc:503] Master 127.8.16.126:36779 was elected leader, sending a full tablet report...
I20260807 05:13:11.038271 26728 raft_consensus.cc:374] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:11.038379 26728 raft_consensus.cc:729] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7112e76ac9574e27918a9e611846ce81, State: Initialized, Role: FOLLOWER
I20260807 05:13:11.038522 26726 ts_tablet_manager.cc:1428] T d1574c5c6822427a8b2921c1f113f5d1 P bc74b2e646be42ccaf5f225c23b0872a: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260807 05:13:11.038602 26727 raft_consensus.cc:348] T d1574c5c6822427a8b2921c1f113f5d1 P 34e83edb4e374c81ad2834a54a7ec0b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc74b2e646be42ccaf5f225c23b0872a" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38633 } } peers { permanent_uuid: "34e83edb4e374c81ad2834a54a7ec0b3" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37307 } } peers { permanent_uuid: "7112e76ac9574e27918a9e611846ce81" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45881 } }
I20260807 05:13:11.038848 26727 raft_consensus.cc:374] T d1574c5c6822427a8b2921c1f113f5d1 P 34e83edb4e374c81ad2834a54a7ec0b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:11.038944 26727 raft_consensus.cc:729] T d1574c5c6822427a8b2921c1f113f5d1 P 34e83edb4e374c81ad2834a54a7ec0b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34e83edb4e374c81ad2834a54a7ec0b3, State: Initialized, Role: FOLLOWER
I20260807 05:13:11.038892 26728 consensus_queue.cc:260] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc74b2e646be42ccaf5f225c23b0872a" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38633 } } peers { permanent_uuid: "34e83edb4e374c81ad2834a54a7ec0b3" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37307 } } peers { permanent_uuid: "7112e76ac9574e27918a9e611846ce81" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45881 } }
I20260807 05:13:11.039489 26408 heartbeater.cc:503] Master 127.8.16.126:36779 was elected leader, sending a full tablet report...
I20260807 05:13:11.039528 26727 consensus_queue.cc:260] T d1574c5c6822427a8b2921c1f113f5d1 P 34e83edb4e374c81ad2834a54a7ec0b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc74b2e646be42ccaf5f225c23b0872a" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38633 } } peers { permanent_uuid: "34e83edb4e374c81ad2834a54a7ec0b3" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37307 } } peers { permanent_uuid: "7112e76ac9574e27918a9e611846ce81" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45881 } }
I20260807 05:13:11.040143 26557 heartbeater.cc:503] Master 127.8.16.126:36779 was elected leader, sending a full tablet report...
I20260807 05:13:11.040447 26727 ts_tablet_manager.cc:1428] T d1574c5c6822427a8b2921c1f113f5d1 P 34e83edb4e374c81ad2834a54a7ec0b3: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260807 05:13:11.041807 26728 ts_tablet_manager.cc:1428] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 05:13:11.202373 26734 raft_consensus.cc:482] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:13:11.202608 26734 raft_consensus.cc:504] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc74b2e646be42ccaf5f225c23b0872a" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38633 } } peers { permanent_uuid: "34e83edb4e374c81ad2834a54a7ec0b3" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37307 } } peers { permanent_uuid: "7112e76ac9574e27918a9e611846ce81" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45881 } }
I20260807 05:13:11.203759 26734 leader_election.cc:288] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc74b2e646be42ccaf5f225c23b0872a (127.8.16.67:38633), 34e83edb4e374c81ad2834a54a7ec0b3 (127.8.16.66:37307)
I20260807 05:13:11.208724 26722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:11.204239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:40503 (local address 127.8.16.66:37307)
0807 05:13:11.204367 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:11.204371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:11.204516 (+ 145us) server_negotiation.cc:407] Connection header received
0807 05:13:11.204615 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:11.204618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:11.204659 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:11.204736 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:11.205044 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.205853 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.207502 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.208115 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.208222 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:11.208286 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:11.208513 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 05:13:11.208581 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 05:13:11.208717 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:11.204153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:58461 (local address 127.8.16.67:38633)
0807 05:13:11.204308 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:11.204312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:11.204434 (+ 122us) server_negotiation.cc:407] Connection header received
0807 05:13:11.204579 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:11.204582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:11.204637 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:11.204719 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:11.205044 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.205852 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.207502 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.208114 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.208227 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:11.208286 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:11.208517 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 05:13:11.208581 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 05:13:11.208784 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:11.204049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:37307 (local address 127.8.16.65:40503)
0807 05:13:11.204439 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:11.204451 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:11.204531 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:11.204747 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:11.204750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:11.204775 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:11.204950 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:11.204956 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.205949 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.205953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:11.207376 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:11.207387 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.208226 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.208229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:11.208391 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:11.208425 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 05:13:11.208494 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 05:13:11.208604 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":224,"spinlock_wait_cycles":768,"thread_start_us":144,"threads_started":1}
I20260807 05:13:11.208787 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:11.203988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:38633 (local address 127.8.16.65:58461)
0807 05:13:11.204335 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:11.204350 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:11.204472 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:11.204747 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:11.204750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:11.204780 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:11.204941 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:11.204948 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.205950 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.205953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:11.207376 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:11.207387 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.208226 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.208229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:11.208379 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:11.208417 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 05:13:11.208506 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 05:13:11.208612 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":78,"spinlock_wait_cycles":15616,"thread_start_us":109,"threads_started":1}
I20260807 05:13:11.209527 26506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1574c5c6822427a8b2921c1f113f5d1" candidate_uuid: "7112e76ac9574e27918a9e611846ce81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34e83edb4e374c81ad2834a54a7ec0b3" is_pre_election: true
I20260807 05:13:11.209646 26654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1574c5c6822427a8b2921c1f113f5d1" candidate_uuid: "7112e76ac9574e27918a9e611846ce81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc74b2e646be42ccaf5f225c23b0872a" is_pre_election: true
I20260807 05:13:11.209848 26506 raft_consensus.cc:2454] T d1574c5c6822427a8b2921c1f113f5d1 P 34e83edb4e374c81ad2834a54a7ec0b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7112e76ac9574e27918a9e611846ce81 in term 0.
I20260807 05:13:11.209921 26654 raft_consensus.cc:2454] T d1574c5c6822427a8b2921c1f113f5d1 P bc74b2e646be42ccaf5f225c23b0872a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7112e76ac9574e27918a9e611846ce81 in term 0.
I20260807 05:13:11.210325 26277 leader_election.cc:302] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 34e83edb4e374c81ad2834a54a7ec0b3, 7112e76ac9574e27918a9e611846ce81, bc74b2e646be42ccaf5f225c23b0872a; no voters:
I20260807 05:13:11.210613 26734 raft_consensus.cc:2781] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:13:11.210700 26734 raft_consensus.cc:482] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:13:11.210752 26734 raft_consensus.cc:3037] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:11.211833 26734 raft_consensus.cc:504] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc74b2e646be42ccaf5f225c23b0872a" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38633 } } peers { permanent_uuid: "34e83edb4e374c81ad2834a54a7ec0b3" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37307 } } peers { permanent_uuid: "7112e76ac9574e27918a9e611846ce81" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45881 } }
I20260807 05:13:11.212152 26734 leader_election.cc:288] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81 [CANDIDATE]: Term 1 election: Requested vote from peers bc74b2e646be42ccaf5f225c23b0872a (127.8.16.67:38633), 34e83edb4e374c81ad2834a54a7ec0b3 (127.8.16.66:37307)
W20260807 05:13:11.212476 26706 tablet.cc:2369] T d1574c5c6822427a8b2921c1f113f5d1 P bc74b2e646be42ccaf5f225c23b0872a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:13:11.212472 26654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1574c5c6822427a8b2921c1f113f5d1" candidate_uuid: "7112e76ac9574e27918a9e611846ce81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc74b2e646be42ccaf5f225c23b0872a"
I20260807 05:13:11.212492 26506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1574c5c6822427a8b2921c1f113f5d1" candidate_uuid: "7112e76ac9574e27918a9e611846ce81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34e83edb4e374c81ad2834a54a7ec0b3"
I20260807 05:13:11.212601 26654 raft_consensus.cc:3037] T d1574c5c6822427a8b2921c1f113f5d1 P bc74b2e646be42ccaf5f225c23b0872a [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:11.212612 26506 raft_consensus.cc:3037] T d1574c5c6822427a8b2921c1f113f5d1 P 34e83edb4e374c81ad2834a54a7ec0b3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:11.213606 26654 raft_consensus.cc:2454] T d1574c5c6822427a8b2921c1f113f5d1 P bc74b2e646be42ccaf5f225c23b0872a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7112e76ac9574e27918a9e611846ce81 in term 1.
I20260807 05:13:11.213768 26506 raft_consensus.cc:2454] T d1574c5c6822427a8b2921c1f113f5d1 P 34e83edb4e374c81ad2834a54a7ec0b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7112e76ac9574e27918a9e611846ce81 in term 1.
I20260807 05:13:11.213892 26279 leader_election.cc:302] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7112e76ac9574e27918a9e611846ce81, bc74b2e646be42ccaf5f225c23b0872a; no voters:
I20260807 05:13:11.214083 26734 raft_consensus.cc:2781] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:11.214308 26734 raft_consensus.cc:686] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81 [term 1 LEADER]: Becoming Leader. State: Replica: 7112e76ac9574e27918a9e611846ce81, State: Running, Role: LEADER
I20260807 05:13:11.214603 26734 consensus_queue.cc:237] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc74b2e646be42ccaf5f225c23b0872a" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38633 } } peers { permanent_uuid: "34e83edb4e374c81ad2834a54a7ec0b3" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37307 } } peers { permanent_uuid: "7112e76ac9574e27918a9e611846ce81" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45881 } }
I20260807 05:13:11.217623 26184 catalog_manager.cc:5495] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7112e76ac9574e27918a9e611846ce81 (127.8.16.65). New cstate: current_term: 1 leader_uuid: "7112e76ac9574e27918a9e611846ce81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc74b2e646be42ccaf5f225c23b0872a" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34e83edb4e374c81ad2834a54a7ec0b3" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7112e76ac9574e27918a9e611846ce81" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 45881 } health_report { overall_health: HEALTHY } } }
I20260807 05:13:11.247949 26725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:11.244384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52388 (local address 127.8.16.65:45881)
0807 05:13:11.244581 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:11.244586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:11.244600 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:13:11.244688 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:11.244690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:11.244743 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:11.244834 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:13:11.245099 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.245650 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.246519 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.246733 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.246820 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:11.247127 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:13:11.247549 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:13:11.247662 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:11.247747 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 05:13:11.247807 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
W20260807 05:13:11.252830 26558 tablet.cc:2369] T d1574c5c6822427a8b2921c1f113f5d1 P 34e83edb4e374c81ad2834a54a7ec0b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:13:11.253763 26654 raft_consensus.cc:1261] T d1574c5c6822427a8b2921c1f113f5d1 P bc74b2e646be42ccaf5f225c23b0872a [term 1 FOLLOWER]: Refusing update from remote peer 7112e76ac9574e27918a9e611846ce81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:13:11.253762 26506 raft_consensus.cc:1261] T d1574c5c6822427a8b2921c1f113f5d1 P 34e83edb4e374c81ad2834a54a7ec0b3 [term 1 FOLLOWER]: Refusing update from remote peer 7112e76ac9574e27918a9e611846ce81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:13:11.254439 26738 consensus_queue.cc:1035] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc74b2e646be42ccaf5f225c23b0872a" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:13:11.254604 26734 consensus_queue.cc:1035] T d1574c5c6822427a8b2921c1f113f5d1 P 7112e76ac9574e27918a9e611846ce81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e83edb4e374c81ad2834a54a7ec0b3" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:13:11.262020 26743 mvcc.cc:205] Tried to move back new op lower bound from 7315782005768155136 to 7315782005617606656. Current Snapshot: MvccSnapshot[applied={T|T < 7315782005768155136}]
I20260807 05:13:11.264863 26742 mvcc.cc:205] Tried to move back new op lower bound from 7315782005768155136 to 7315782005617606656. Current Snapshot: MvccSnapshot[applied={T|T < 7315782005768155136}]
I20260807 05:13:11.271531 26185 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:13:11.271694 26185 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:13:11.273931 26185 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f8737003b18b482ab5bb1ffb6794c741: Sending DeleteTablet for 3 replicas of tablet d1574c5c6822427a8b2921c1f113f5d1
I20260807 05:13:11.274776 26632 tablet_service.cc:1514] Processing DeleteTablet for tablet d1574c5c6822427a8b2921c1f113f5d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:13:11 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:43586
I20260807 05:13:11.274806 26485 tablet_service.cc:1514] Processing DeleteTablet for tablet d1574c5c6822427a8b2921c1f113f5d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:13:11 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:60548
I20260807 05:13:11.275305 26752 tablet_replica.cc:330] T d1574c5c6822427a8b2921c1f113f5d1 P bc74b2e646be42ccaf5f225c23b0872a: stopping tablet replica
I20260807 05:13:11.275276 26331 tablet_service.cc:1514] Processing DeleteTablet for tablet d1574c5c6822427a8b2921c1f113f5d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:13:11 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:52384
I20260807 05:13:11.275705 26752 raft_consensus.cc:2229] T d1574c5c6822427a8b2921c1f113f5d1 P bc74b2e646be42ccaf5f225c23b0872a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:13:11.276064 26752 raft_consensus.cc:2258] T d1574c5c6822427a8b2921c1f113f5d1 P bc74b2e646be42ccaf5f225c23b0872a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:13:11.276631 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 26258
I20260807 05:13:11.278712 26752 ts_tablet_manager.cc:1905] T d1574c5c6822427a8b2921c1f113f5d1 P bc74b2e646be42ccaf5f225c23b0872a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:13:11.279464 26751 tablet_replica.cc:330] T d1574c5c6822427a8b2921c1f113f5d1 P 34e83edb4e374c81ad2834a54a7ec0b3: stopping tablet replica
I20260807 05:13:11.279772 26751 raft_consensus.cc:2229] T d1574c5c6822427a8b2921c1f113f5d1 P 34e83edb4e374c81ad2834a54a7ec0b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:13:11.280047 26751 raft_consensus.cc:2258] T d1574c5c6822427a8b2921c1f113f5d1 P 34e83edb4e374c81ad2834a54a7ec0b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:13:11.281430 26751 ts_tablet_manager.cc:1905] T d1574c5c6822427a8b2921c1f113f5d1 P 34e83edb4e374c81ad2834a54a7ec0b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:13:11.281536 26752 ts_tablet_manager.cc:1918] T d1574c5c6822427a8b2921c1f113f5d1 P bc74b2e646be42ccaf5f225c23b0872a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:13:11.281632 26752 log.cc:1196] T d1574c5c6822427a8b2921c1f113f5d1 P bc74b2e646be42ccaf5f225c23b0872a: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/d1574c5c6822427a8b2921c1f113f5d1
I20260807 05:13:11.281941 26752 ts_tablet_manager.cc:1939] T d1574c5c6822427a8b2921c1f113f5d1 P bc74b2e646be42ccaf5f225c23b0872a: Deleting consensus metadata
I20260807 05:13:11.283142 26173 catalog_manager.cc:4841] TS bc74b2e646be42ccaf5f225c23b0872a (127.8.16.67:38633): tablet d1574c5c6822427a8b2921c1f113f5d1 (table test-table [id=1884a95964094705afca6bdfbd0a3f07]) successfully deleted
I20260807 05:13:11.283604 26751 ts_tablet_manager.cc:1918] T d1574c5c6822427a8b2921c1f113f5d1 P 34e83edb4e374c81ad2834a54a7ec0b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:13:11.283686 26751 log.cc:1196] T d1574c5c6822427a8b2921c1f113f5d1 P 34e83edb4e374c81ad2834a54a7ec0b3: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079354071073-8257-0/minicluster-data/ts-1/wal/wals/d1574c5c6822427a8b2921c1f113f5d1
I20260807 05:13:11.283959 26751 ts_tablet_manager.cc:1939] T d1574c5c6822427a8b2921c1f113f5d1 P 34e83edb4e374c81ad2834a54a7ec0b3: Deleting consensus metadata
W20260807 05:13:11.284583 26173 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.16.65:45881 (error 108)
I20260807 05:13:11.284787 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 26413
I20260807 05:13:11.286008 26723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:11.285721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:45881 (local address 127.0.0.1:52398)
0807 05:13:11.285842 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:11.285929 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.65:45881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 05:13:11.286242 26173 catalog_manager.cc:4584] TS 7112e76ac9574e27918a9e611846ce81 (127.8.16.65:45881): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d1574c5c6822427a8b2921c1f113f5d1: Network error: Client connection negotiation failed: client connection to 127.8.16.65:45881: connect: Connection refused (error 111)
I20260807 05:13:11.292676 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 26562
I20260807 05:13:11.294140 26723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:11.293790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:37307 (local address 127.0.0.1:60560)
0807 05:13:11.293975 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:11.294040 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.66:37307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260807 05:13:11.298676 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 26152
2026-08-07T05:13:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2321 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T05:13:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:11Z Disabled control of system clock
I20260807 05:13:11.328682 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:45429
--webserver_interface=127.8.16.126
--webserver_port=0
--builtin_ntp_servers=127.8.16.84:35247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:45429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 05:13:11.436468 26760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:11.436794 26760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:11.436836 26760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:11.440768 26760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:11.440826 26760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:11.440840 26760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:11.440856 26760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:11.440871 26760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:11.445546 26760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:35247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:45429
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:45429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:11.446627 26760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:11.447952 26760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:11.455914 26766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:11.455979 26767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:11.455967 26769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:11.456239 26760 server_base.cc:1034] running on GCE node
I20260807 05:13:11.456864 26760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:11.458009 26760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:11.459196 26760 hybrid_clock.cc:648] HybridClock initialized: now 1786079591459182 us; error 26 us; skew 500 ppm
I20260807 05:13:11.460984 26760 webserver.cc:460] Webserver started at http://127.8.16.126:43351/ using document root <none> and password file <none>
I20260807 05:13:11.461597 26760 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:11.461673 26760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:11.461871 26760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:11.463915 26760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "8a681e82c7a24e90855e4b4804be535e"
format_stamp: "Formatted at 2026-08-07 05:13:11 on dist-test-slave-49h3"
server_key: "d5f283efb606ce8d4929a7325e88742b"
server_key_iv: "4781e98527199104bb37670ab3ddd7ad"
server_key_version: "encryptionkey@0"
I20260807 05:13:11.464460 26760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "8a681e82c7a24e90855e4b4804be535e"
format_stamp: "Formatted at 2026-08-07 05:13:11 on dist-test-slave-49h3"
server_key: "d5f283efb606ce8d4929a7325e88742b"
server_key_iv: "4781e98527199104bb37670ab3ddd7ad"
server_key_version: "encryptionkey@0"
I20260807 05:13:11.468055 26760 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:13:11.470420 26776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:11.471589 26760 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:13:11.471724 26760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "8a681e82c7a24e90855e4b4804be535e"
format_stamp: "Formatted at 2026-08-07 05:13:11 on dist-test-slave-49h3"
server_key: "d5f283efb606ce8d4929a7325e88742b"
server_key_iv: "4781e98527199104bb37670ab3ddd7ad"
server_key_version: "encryptionkey@0"
I20260807 05:13:11.471819 26760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:11.507203 26760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:11.507951 26760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:11.508174 26760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:11.515074 26760 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:45429
I20260807 05:13:11.515082 26833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:45429 every 8 connection(s)
I20260807 05:13:11.516013 26760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:11.519032 26834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:11.524555 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 26760
I20260807 05:13:11.524688 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:11.524755 26834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e: Bootstrap starting.
I20260807 05:13:11.525010 8257 external_mini_cluster.cc:1428] Setting key ffd8a9c59c2ce4a763038d1874a25e01
I20260807 05:13:11.527441 26834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:11.528157 26834 log.cc:824] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:11.530041 26834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e: No bootstrap required, opened a new log
I20260807 05:13:11.533663 26834 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a681e82c7a24e90855e4b4804be535e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 45429 } }
I20260807 05:13:11.533895 26834 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:11.533952 26834 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a681e82c7a24e90855e4b4804be535e, State: Initialized, Role: FOLLOWER
I20260807 05:13:11.534121 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:11.526439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50406 (local address 127.8.16.126:45429)
0807 05:13:11.527102 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:11.527116 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:11.527157 (+ 41us) server_negotiation.cc:407] Connection header received
0807 05:13:11.527838 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:11.527856 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:11.528264 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:11.528574 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:11.528997 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.529983 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.530887 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.531361 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.531508 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:11.532351 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:11.532357 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:11.532370 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:11.532745 (+ 375us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:11.532801 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:11.532820 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:11.532904 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:11.533042 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 05:13:11.533300 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":147,"threads_started":1}
I20260807 05:13:11.534412 26834 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a681e82c7a24e90855e4b4804be535e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 45429 } }
I20260807 05:13:11.534577 26834 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:11.534643 26834 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:11.534735 26834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:11.535712 26834 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a681e82c7a24e90855e4b4804be535e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 45429 } }
I20260807 05:13:11.536027 26834 leader_election.cc:302] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a681e82c7a24e90855e4b4804be535e; no voters:
I20260807 05:13:11.536367 26834 leader_election.cc:288] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:11.536471 26839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:11.536686 26839 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e [term 1 LEADER]: Becoming Leader. State: Replica: 8a681e82c7a24e90855e4b4804be535e, State: Running, Role: LEADER
I20260807 05:13:11.537011 26839 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a681e82c7a24e90855e4b4804be535e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 45429 } }
I20260807 05:13:11.537513 26834 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:11.538559 26841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a681e82c7a24e90855e4b4804be535e. Latest consensus state: current_term: 1 leader_uuid: "8a681e82c7a24e90855e4b4804be535e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a681e82c7a24e90855e4b4804be535e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 45429 } } }
I20260807 05:13:11.538688 26841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:11.538945 26840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a681e82c7a24e90855e4b4804be535e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a681e82c7a24e90855e4b4804be535e" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 45429 } } }
I20260807 05:13:11.539047 26848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:11.539126 26840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:11.542120 26848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:11.547847 26848 catalog_manager.cc:1347] Generated new cluster ID: 62de9dcbd0ca4a4786508bbcbeba8ae9
I20260807 05:13:11.547979 26848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:11.553296 26848 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:11.554214 26848 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:11.562263 26848 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e: Generated new TSK 0
I20260807 05:13:11.562938 26848 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:13:11.571751 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:45429
--builtin_ntp_servers=127.8.16.84:35247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 05:13:11.700994 26860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:11.701299 26860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:11.701359 26860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:11.705518 26860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:11.705590 26860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:11.705668 26860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:13:11.710564 26860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:35247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:45429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:11.711654 26860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:11.713001 26860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:11.722069 26869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:11.722071 26867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:11.722071 26865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:11.722397 26860 server_base.cc:1034] running on GCE node
I20260807 05:13:11.722810 26860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:11.723426 26860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:11.724586 26860 hybrid_clock.cc:648] HybridClock initialized: now 1786079591724569 us; error 29 us; skew 500 ppm
I20260807 05:13:11.726398 26860 webserver.cc:460] Webserver started at http://127.8.16.65:39833/ using document root <none> and password file <none>
I20260807 05:13:11.726963 26860 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:11.727030 26860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:11.727237 26860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:11.729348 26860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "151ff8be277a442b93c50c965e92a8cc"
format_stamp: "Formatted at 2026-08-07 05:13:11 on dist-test-slave-49h3"
server_key: "b87d37059d47340b0c6a78761fbb8872"
server_key_iv: "9a29acf272765ab7276880cb68ff6c87"
server_key_version: "encryptionkey@0"
I20260807 05:13:11.729930 26860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "151ff8be277a442b93c50c965e92a8cc"
format_stamp: "Formatted at 2026-08-07 05:13:11 on dist-test-slave-49h3"
server_key: "b87d37059d47340b0c6a78761fbb8872"
server_key_iv: "9a29acf272765ab7276880cb68ff6c87"
server_key_version: "encryptionkey@0"
I20260807 05:13:11.733435 26860 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:13:11.735870 26875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:11.737030 26860 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 05:13:11.737138 26860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "151ff8be277a442b93c50c965e92a8cc"
format_stamp: "Formatted at 2026-08-07 05:13:11 on dist-test-slave-49h3"
server_key: "b87d37059d47340b0c6a78761fbb8872"
server_key_iv: "9a29acf272765ab7276880cb68ff6c87"
server_key_version: "encryptionkey@0"
I20260807 05:13:11.737238 26860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:11.748245 26860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:11.749006 26860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:11.749189 26860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:11.749850 26860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:13:11.750818 26860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:13:11.750869 26860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:11.750921 26860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:13:11.750950 26860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:11.759335 26860 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:41691
I20260807 05:13:11.759372 27000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:41691 every 8 connection(s)
I20260807 05:13:11.760223 26860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:13:11.768642 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:11.761986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:52753 (local address 127.8.16.126:45429)
0807 05:13:11.762128 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:11.762133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:11.763200 (+ 1067us) server_negotiation.cc:407] Connection header received
0807 05:13:11.764496 (+ 1296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:11.764499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:11.764544 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:11.764616 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:11.765425 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.765991 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.767053 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.767266 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.767334 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:11.768267 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:11.768270 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:11.768272 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:11.768285 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:11.768306 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:11.768313 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:11.768365 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:11.768470 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 05:13:11.768523 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 05:13:11.769469 27004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:11.762414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:45429 (local address 127.8.16.65:52753)
0807 05:13:11.763060 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:11.763101 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:11.764327 (+ 1226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:11.764769 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:11.764775 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:11.764854 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:11.765318 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:11.765331 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.766093 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.766095 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:11.766964 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:11.766973 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.767372 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.767374 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:11.767505 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:11.768030 (+ 525us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:13:11.768032 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:11.768070 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:13:11.768131 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:11.768371 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:11.768376 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:11.768379 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 05:13:11.768462 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 05:13:11.768673 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":128,"threads_started":1}
I20260807 05:13:11.769521 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 26860
I20260807 05:13:11.769618 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:13:11.769924 8257 external_mini_cluster.cc:1428] Setting key 92571d2fb76d1e212640525c3591a258
I20260807 05:13:11.770709 27001 heartbeater.cc:347] Connected to a master server at 127.8.16.126:45429
I20260807 05:13:11.770952 27001 heartbeater.cc:464] Registering TS with master...
I20260807 05:13:11.771510 27001 heartbeater.cc:511] Master 127.8.16.126:45429 requested a full tablet report, sending...
I20260807 05:13:11.771925 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:45429
--builtin_ntp_servers=127.8.16.84:35247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 05:13:11.773095 26795 ts_manager.cc:194] Registered new tserver with Master: 151ff8be277a442b93c50c965e92a8cc (127.8.16.65:41691)
I20260807 05:13:11.774344 26795 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.16.65:52753
W20260807 05:13:11.879549 27005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:11.879802 27005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:11.879841 27005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:11.883818 27005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:11.883883 27005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:11.883947 27005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:13:11.888636 27005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:35247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:45429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:11.889698 27005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:11.891036 27005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:11.900316 27011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:11.900296 27012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:11.900346 27005 server_base.cc:1034] running on GCE node
W20260807 05:13:11.900296 27014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:11.900900 27005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:11.901548 27005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:11.902698 27005 hybrid_clock.cc:648] HybridClock initialized: now 1786079591902674 us; error 42 us; skew 500 ppm
I20260807 05:13:11.904445 27005 webserver.cc:460] Webserver started at http://127.8.16.66:46149/ using document root <none> and password file <none>
I20260807 05:13:11.904996 27005 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:11.905066 27005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:11.905257 27005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:11.907303 27005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "93791b23b53f4331857f04281bc6b4b3"
format_stamp: "Formatted at 2026-08-07 05:13:11 on dist-test-slave-49h3"
server_key: "d9f63cf9a27f39b9aafaa5ebdf982d73"
server_key_iv: "b87ca88918c8ec57dab1b865d81910c7"
server_key_version: "encryptionkey@0"
I20260807 05:13:11.907827 27005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "93791b23b53f4331857f04281bc6b4b3"
format_stamp: "Formatted at 2026-08-07 05:13:11 on dist-test-slave-49h3"
server_key: "d9f63cf9a27f39b9aafaa5ebdf982d73"
server_key_iv: "b87ca88918c8ec57dab1b865d81910c7"
server_key_version: "encryptionkey@0"
I20260807 05:13:11.911391 27005 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:13:11.913658 27021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:11.914780 27005 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:13:11.914884 27005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "93791b23b53f4331857f04281bc6b4b3"
format_stamp: "Formatted at 2026-08-07 05:13:11 on dist-test-slave-49h3"
server_key: "d9f63cf9a27f39b9aafaa5ebdf982d73"
server_key_iv: "b87ca88918c8ec57dab1b865d81910c7"
server_key_version: "encryptionkey@0"
I20260807 05:13:11.914976 27005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:11.933880 27005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:11.934602 27005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:11.934815 27005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:11.935552 27005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:13:11.936745 27005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:13:11.936806 27005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:11.936862 27005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:13:11.936929 27005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:11.946823 27005 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:46333
I20260807 05:13:11.946868 27153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:46333 every 8 connection(s)
I20260807 05:13:11.947737 27005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:13:11.948853 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 27005
I20260807 05:13:11.948966 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:13:11.949249 8257 external_mini_cluster.cc:1428] Setting key f3dc16d38855139380d08fc1f5b20759
I20260807 05:13:11.951332 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:45429
--builtin_ntp_servers=127.8.16.84:35247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 05:13:11.957142 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:11.949599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:36641 (local address 127.8.16.126:45429)
0807 05:13:11.949736 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:11.949740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:11.950830 (+ 1090us) server_negotiation.cc:407] Connection header received
0807 05:13:11.952247 (+ 1417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:11.952251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:11.952303 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:11.952388 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:11.953493 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.954054 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.955224 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.955549 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.955657 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:11.956658 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:11.956662 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:11.956664 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:11.956680 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:11.956705 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:11.956716 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:11.956788 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:11.956915 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 05:13:11.956979 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 05:13:11.958036 27156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:11.949932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:45429 (local address 127.8.16.66:36641)
0807 05:13:11.950661 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:11.950718 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:11.952020 (+ 1302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:11.952535 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:11.952542 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:11.952628 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:11.953321 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:11.953339 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.954213 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.954216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:11.955094 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:11.955103 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:11.955692 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:11.955695 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:11.955840 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:11.956402 (+ 562us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:13:11.956404 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:11.956444 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:13:11.956510 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:11.956797 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:11.956802 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:11.956806 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 05:13:11.956893 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 05:13:11.957117 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":121,"threads_started":1}
I20260807 05:13:11.959131 27154 heartbeater.cc:347] Connected to a master server at 127.8.16.126:45429
I20260807 05:13:11.959404 27154 heartbeater.cc:464] Registering TS with master...
I20260807 05:13:11.960035 27154 heartbeater.cc:511] Master 127.8.16.126:45429 requested a full tablet report, sending...
I20260807 05:13:11.961135 26795 ts_manager.cc:194] Registered new tserver with Master: 93791b23b53f4331857f04281bc6b4b3 (127.8.16.66:46333)
I20260807 05:13:11.961740 26795 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.16.66:36641
W20260807 05:13:12.063632 27157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:12.063876 27157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:12.063918 27157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:12.067912 27157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:12.067972 27157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:12.068040 27157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:13:12.072714 27157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:35247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:45429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:12.073776 27157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:12.075093 27157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:12.084192 27162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:12.084199 27165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:12.084199 27163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:12.084481 27157 server_base.cc:1034] running on GCE node
I20260807 05:13:12.084851 27157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:12.085409 27157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:12.086575 27157 hybrid_clock.cc:648] HybridClock initialized: now 1786079592086560 us; error 29 us; skew 500 ppm
I20260807 05:13:12.088307 27157 webserver.cc:460] Webserver started at http://127.8.16.67:36861/ using document root <none> and password file <none>
I20260807 05:13:12.088822 27157 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:12.088878 27157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:12.089043 27157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:12.091027 27157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "1c6243b47d244f5ba25b97daa032b943"
format_stamp: "Formatted at 2026-08-07 05:13:12 on dist-test-slave-49h3"
server_key: "473b9182ad282c6f0515bdba52b8b20b"
server_key_iv: "f831dfa10d3169773f320eae26f5797a"
server_key_version: "encryptionkey@0"
I20260807 05:13:12.091523 27157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "1c6243b47d244f5ba25b97daa032b943"
format_stamp: "Formatted at 2026-08-07 05:13:12 on dist-test-slave-49h3"
server_key: "473b9182ad282c6f0515bdba52b8b20b"
server_key_iv: "f831dfa10d3169773f320eae26f5797a"
server_key_version: "encryptionkey@0"
I20260807 05:13:12.094985 27157 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:13:12.097301 27171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:12.098503 27157 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:13:12.098601 27157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "1c6243b47d244f5ba25b97daa032b943"
format_stamp: "Formatted at 2026-08-07 05:13:12 on dist-test-slave-49h3"
server_key: "473b9182ad282c6f0515bdba52b8b20b"
server_key_iv: "f831dfa10d3169773f320eae26f5797a"
server_key_version: "encryptionkey@0"
I20260807 05:13:12.098688 27157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:12.122004 27157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:12.122694 27157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:12.122848 27157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:12.123414 27157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:13:12.124347 27157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:13:12.124394 27157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:12.124432 27157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:13:12.124452 27157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:12.132845 27157 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:45181
I20260807 05:13:12.132870 27296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:45181 every 8 connection(s)
I20260807 05:13:12.133777 27157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:13:12.138156 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 27157
I20260807 05:13:12.138269 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:13:12.138523 8257 external_mini_cluster.cc:1428] Setting key 6d11bba8870206452f3f979078929821
I20260807 05:13:12.142952 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:12.135521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:34247 (local address 127.8.16.126:45429)
0807 05:13:12.135671 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:12.135676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:12.136731 (+ 1055us) server_negotiation.cc:407] Connection header received
0807 05:13:12.138146 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:12.138149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:12.138202 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:12.138293 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:12.139357 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.140074 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.141280 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.141573 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.141653 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:12.142594 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:12.142597 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:12.142599 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:12.142611 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:12.142630 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:12.142637 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:12.142690 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:12.142786 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 05:13:12.142836 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 05:13:12.143757 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:12.135918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:45429 (local address 127.8.16.67:34247)
0807 05:13:12.136571 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:12.136612 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:12.137933 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:12.138463 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:12.138470 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:12.138554 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:12.139216 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:12.139231 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.140202 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.140205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:12.141173 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:12.141183 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.141693 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.141696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:12.141847 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:12.142358 (+ 511us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:13:12.142360 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:12.142401 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:13:12.142465 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:12.142690 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:12.142695 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:12.142698 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 05:13:12.142782 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 05:13:12.142995 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":118,"threads_started":1}
I20260807 05:13:12.144733 27297 heartbeater.cc:347] Connected to a master server at 127.8.16.126:45429
I20260807 05:13:12.144960 27297 heartbeater.cc:464] Registering TS with master...
I20260807 05:13:12.145473 27297 heartbeater.cc:511] Master 127.8.16.126:45429 requested a full tablet report, sending...
I20260807 05:13:12.146430 26795 ts_manager.cc:194] Registered new tserver with Master: 1c6243b47d244f5ba25b97daa032b943 (127.8.16.67:45181)
I20260807 05:13:12.146977 26795 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.16.67:34247
I20260807 05:13:12.152245 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:13:12.157621 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:12.154099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50416 (local address 127.8.16.126:45429)
0807 05:13:12.154221 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:12.154226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:12.154418 (+ 192us) server_negotiation.cc:407] Connection header received
0807 05:13:12.154525 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:12.154528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:12.154574 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:12.154638 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:12.154908 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.155407 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.156198 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.156418 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.156502 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:12.157224 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:12.157227 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:12.157230 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:12.157243 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:12.157263 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:12.157270 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:12.157327 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:12.157414 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 05:13:12.157488 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:13:12.160066 26795 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:13:12.162979 26795 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:13:12.176252 27313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:12.171257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52524 (local address 127.8.16.66:46333)
0807 05:13:12.171897 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:12.171908 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:12.171930 (+ 22us) server_negotiation.cc:407] Connection header received
0807 05:13:12.172006 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:12.172010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:12.172182 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:12.172302 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:12.172525 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.173378 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.175064 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.175616 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.175716 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:12.175916 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:12.176002 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 05:13:12.176091 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":133,"threads_started":1}
I20260807 05:13:12.176280 27309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:12.170566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:46333 (local address 127.0.0.1:52524)
0807 05:13:12.171369 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:12.171382 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:12.171464 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:12.172290 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:12.172293 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:12.172311 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:12.172396 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:12.172402 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.173493 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.173496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:12.174958 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:12.174965 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.175720 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.175724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:12.175838 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:12.175874 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 05:13:12.175975 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 05:13:12.176086 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"mutex_wait_us":51,"thread_start_us":80,"threads_started":1}
I20260807 05:13:12.176468 27308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:12.170466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:41691 (local address 127.0.0.1:52846)
0807 05:13:12.170821 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:12.170871 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:12.170990 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:12.171969 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:12.171975 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:12.171999 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:12.172144 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:12.172151 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.173514 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.173518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:12.174809 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:12.174818 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.176069 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.176072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:12.176166 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:12.176190 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 05:13:12.176254 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 05:13:12.176324 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":48,"thread_start_us":79,"threads_started":1}
I20260807 05:13:12.176868 27310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:12.170596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52846 (local address 127.8.16.65:41691)
0807 05:13:12.171499 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:12.171512 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:12.171536 (+ 24us) server_negotiation.cc:407] Connection header received
0807 05:13:12.171604 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:12.171608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:12.171819 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:12.171997 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:12.172254 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.173376 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.174929 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.175734 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.175831 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:12.176002 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:12.176620 (+ 618us) server_negotiation.cc:299] Negotiation successful
0807 05:13:12.176715 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":795,"thread_start_us":103,"threads_started":1}
I20260807 05:13:12.177062 27312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:12.171216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:45181 (local address 127.0.0.1:46786)
0807 05:13:12.172491 (+ 1275us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:12.172504 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:12.172598 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:12.172996 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:12.172999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:12.173016 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:12.173090 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:12.173095 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.174264 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.174267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:12.175228 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:12.175239 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.176699 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.176702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:12.176804 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:12.176825 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 05:13:12.176876 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 05:13:12.176939 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1214,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
I20260807 05:13:12.177567 27314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:12.171633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46786 (local address 127.8.16.67:45181)
0807 05:13:12.172088 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:12.172099 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:12.172599 (+ 500us) server_negotiation.cc:407] Connection header received
0807 05:13:12.172697 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:12.172702 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:12.172864 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:12.173001 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:12.173177 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.174143 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.175368 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.176091 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.176226 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:12.177095 (+ 869us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:12.177290 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 05:13:12.177386 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":165,"threads_started":1}
I20260807 05:13:12.179240 26926 tablet_service.cc:1467] Processing CreateTablet for tablet d0fa289b20bd4f4999803bcf9ba2ed59 (DEFAULT_TABLE table=test-table [id=5d0afc3e329b4ffab74fc3835452cec4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:13:12.179242 27073 tablet_service.cc:1467] Processing CreateTablet for tablet d0fa289b20bd4f4999803bcf9ba2ed59 (DEFAULT_TABLE table=test-table [id=5d0afc3e329b4ffab74fc3835452cec4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:13:12.180166 26926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0fa289b20bd4f4999803bcf9ba2ed59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:12.180224 27073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0fa289b20bd4f4999803bcf9ba2ed59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:12.180032 27224 tablet_service.cc:1467] Processing CreateTablet for tablet d0fa289b20bd4f4999803bcf9ba2ed59 (DEFAULT_TABLE table=test-table [id=5d0afc3e329b4ffab74fc3835452cec4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:13:12.180941 27224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0fa289b20bd4f4999803bcf9ba2ed59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:12.187165 27316 tablet_bootstrap.cc:492] T d0fa289b20bd4f4999803bcf9ba2ed59 P 93791b23b53f4331857f04281bc6b4b3: Bootstrap starting.
I20260807 05:13:12.187536 27315 tablet_bootstrap.cc:492] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc: Bootstrap starting.
I20260807 05:13:12.189177 27316 tablet_bootstrap.cc:654] T d0fa289b20bd4f4999803bcf9ba2ed59 P 93791b23b53f4331857f04281bc6b4b3: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:12.189245 27315 tablet_bootstrap.cc:654] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:12.189700 27317 tablet_bootstrap.cc:492] T d0fa289b20bd4f4999803bcf9ba2ed59 P 1c6243b47d244f5ba25b97daa032b943: Bootstrap starting.
I20260807 05:13:12.189931 27316 log.cc:824] T d0fa289b20bd4f4999803bcf9ba2ed59 P 93791b23b53f4331857f04281bc6b4b3: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:12.190038 27315 log.cc:824] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:12.191689 27317 tablet_bootstrap.cc:654] T d0fa289b20bd4f4999803bcf9ba2ed59 P 1c6243b47d244f5ba25b97daa032b943: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:12.191728 27315 tablet_bootstrap.cc:492] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc: No bootstrap required, opened a new log
I20260807 05:13:12.191887 27316 tablet_bootstrap.cc:492] T d0fa289b20bd4f4999803bcf9ba2ed59 P 93791b23b53f4331857f04281bc6b4b3: No bootstrap required, opened a new log
I20260807 05:13:12.191905 27315 ts_tablet_manager.cc:1397] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 05:13:12.192062 27316 ts_tablet_manager.cc:1397] T d0fa289b20bd4f4999803bcf9ba2ed59 P 93791b23b53f4331857f04281bc6b4b3: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 05:13:12.192487 27317 log.cc:824] T d0fa289b20bd4f4999803bcf9ba2ed59 P 1c6243b47d244f5ba25b97daa032b943: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:12.194540 27317 tablet_bootstrap.cc:492] T d0fa289b20bd4f4999803bcf9ba2ed59 P 1c6243b47d244f5ba25b97daa032b943: No bootstrap required, opened a new log
I20260807 05:13:12.194716 27317 ts_tablet_manager.cc:1397] T d0fa289b20bd4f4999803bcf9ba2ed59 P 1c6243b47d244f5ba25b97daa032b943: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:13:12.195349 27315 raft_consensus.cc:348] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "151ff8be277a442b93c50c965e92a8cc" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 41691 } } peers { permanent_uuid: "93791b23b53f4331857f04281bc6b4b3" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46333 } } peers { permanent_uuid: "1c6243b47d244f5ba25b97daa032b943" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 45181 } }
I20260807 05:13:12.195561 27315 raft_consensus.cc:374] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:12.195612 27315 raft_consensus.cc:729] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 151ff8be277a442b93c50c965e92a8cc, State: Initialized, Role: FOLLOWER
I20260807 05:13:12.196004 27315 consensus_queue.cc:260] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "151ff8be277a442b93c50c965e92a8cc" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 41691 } } peers { permanent_uuid: "93791b23b53f4331857f04281bc6b4b3" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46333 } } peers { permanent_uuid: "1c6243b47d244f5ba25b97daa032b943" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 45181 } }
I20260807 05:13:12.196125 27316 raft_consensus.cc:348] T d0fa289b20bd4f4999803bcf9ba2ed59 P 93791b23b53f4331857f04281bc6b4b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "151ff8be277a442b93c50c965e92a8cc" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 41691 } } peers { permanent_uuid: "93791b23b53f4331857f04281bc6b4b3" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46333 } } peers { permanent_uuid: "1c6243b47d244f5ba25b97daa032b943" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 45181 } }
I20260807 05:13:12.196359 27316 raft_consensus.cc:374] T d0fa289b20bd4f4999803bcf9ba2ed59 P 93791b23b53f4331857f04281bc6b4b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:12.196413 27316 raft_consensus.cc:729] T d0fa289b20bd4f4999803bcf9ba2ed59 P 93791b23b53f4331857f04281bc6b4b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93791b23b53f4331857f04281bc6b4b3, State: Initialized, Role: FOLLOWER
I20260807 05:13:12.196728 27001 heartbeater.cc:503] Master 127.8.16.126:45429 was elected leader, sending a full tablet report...
I20260807 05:13:12.196867 27316 consensus_queue.cc:260] T d0fa289b20bd4f4999803bcf9ba2ed59 P 93791b23b53f4331857f04281bc6b4b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "151ff8be277a442b93c50c965e92a8cc" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 41691 } } peers { permanent_uuid: "93791b23b53f4331857f04281bc6b4b3" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46333 } } peers { permanent_uuid: "1c6243b47d244f5ba25b97daa032b943" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 45181 } }
I20260807 05:13:12.196959 27315 ts_tablet_manager.cc:1428] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260807 05:13:12.197755 27316 ts_tablet_manager.cc:1428] T d0fa289b20bd4f4999803bcf9ba2ed59 P 93791b23b53f4331857f04281bc6b4b3: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 05:13:12.198094 27154 heartbeater.cc:503] Master 127.8.16.126:45429 was elected leader, sending a full tablet report...
W20260807 05:13:12.198602 27155 tablet.cc:2369] T d0fa289b20bd4f4999803bcf9ba2ed59 P 93791b23b53f4331857f04281bc6b4b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:13:12.198630 27317 raft_consensus.cc:348] T d0fa289b20bd4f4999803bcf9ba2ed59 P 1c6243b47d244f5ba25b97daa032b943 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "151ff8be277a442b93c50c965e92a8cc" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 41691 } } peers { permanent_uuid: "93791b23b53f4331857f04281bc6b4b3" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46333 } } peers { permanent_uuid: "1c6243b47d244f5ba25b97daa032b943" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 45181 } }
I20260807 05:13:12.199170 27317 raft_consensus.cc:374] T d0fa289b20bd4f4999803bcf9ba2ed59 P 1c6243b47d244f5ba25b97daa032b943 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:12.199245 27317 raft_consensus.cc:729] T d0fa289b20bd4f4999803bcf9ba2ed59 P 1c6243b47d244f5ba25b97daa032b943 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c6243b47d244f5ba25b97daa032b943, State: Initialized, Role: FOLLOWER
I20260807 05:13:12.199666 27317 consensus_queue.cc:260] T d0fa289b20bd4f4999803bcf9ba2ed59 P 1c6243b47d244f5ba25b97daa032b943 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "151ff8be277a442b93c50c965e92a8cc" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 41691 } } peers { permanent_uuid: "93791b23b53f4331857f04281bc6b4b3" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46333 } } peers { permanent_uuid: "1c6243b47d244f5ba25b97daa032b943" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 45181 } }
I20260807 05:13:12.200378 27297 heartbeater.cc:503] Master 127.8.16.126:45429 was elected leader, sending a full tablet report...
I20260807 05:13:12.200563 27317 ts_tablet_manager.cc:1428] T d0fa289b20bd4f4999803bcf9ba2ed59 P 1c6243b47d244f5ba25b97daa032b943: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260807 05:13:12.261317 27003 tablet.cc:2369] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:13:12.263406 27321 raft_consensus.cc:482] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:13:12.263573 27321 raft_consensus.cc:504] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "151ff8be277a442b93c50c965e92a8cc" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 41691 } } peers { permanent_uuid: "93791b23b53f4331857f04281bc6b4b3" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46333 } } peers { permanent_uuid: "1c6243b47d244f5ba25b97daa032b943" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 45181 } }
I20260807 05:13:12.264642 27321 leader_election.cc:288] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93791b23b53f4331857f04281bc6b4b3 (127.8.16.66:46333), 1c6243b47d244f5ba25b97daa032b943 (127.8.16.67:45181)
I20260807 05:13:12.268872 27313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:12.265081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:39193 (local address 127.8.16.66:46333)
0807 05:13:12.265191 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:12.265193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:12.265215 (+ 22us) server_negotiation.cc:407] Connection header received
0807 05:13:12.265354 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:12.265357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:12.265417 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:12.265513 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:12.265784 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.266608 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.267936 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.268327 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.268416 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:12.268476 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:12.268678 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 05:13:12.268746 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:13:12.268873 27314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:12.264992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:48049 (local address 127.8.16.67:45181)
0807 05:13:12.265132 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:12.265137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:12.265355 (+ 218us) server_negotiation.cc:407] Connection header received
0807 05:13:12.265481 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:12.265484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:12.265535 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:12.265592 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:12.265839 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.266581 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.267934 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.268330 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.268425 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:12.268485 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:12.268668 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 05:13:12.268749 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 05:13:12.268877 27004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:12.264895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:46333 (local address 127.8.16.65:39193)
0807 05:13:12.265099 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:12.265115 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:12.265210 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:12.265522 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:12.265524 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:12.265540 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:12.265659 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:12.265667 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.266714 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.266717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:12.267820 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:12.267830 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.268453 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.268457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:12.268566 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:12.268585 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 05:13:12.268650 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 05:13:12.268755 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":92}
I20260807 05:13:12.268893 27324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:12.264895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:45181 (local address 127.8.16.65:48049)
0807 05:13:12.265266 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:12.265279 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:12.265359 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:12.265624 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:12.265627 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:12.265640 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:12.265746 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:12.265752 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.266731 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.266735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:12.267819 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:12.267829 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.268441 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.268445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:12.268556 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:12.268580 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 05:13:12.268643 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 05:13:12.268743 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":74,"thread_start_us":88,"threads_started":1}
I20260807 05:13:12.269519 27101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0fa289b20bd4f4999803bcf9ba2ed59" candidate_uuid: "151ff8be277a442b93c50c965e92a8cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93791b23b53f4331857f04281bc6b4b3" is_pre_election: true
I20260807 05:13:12.269770 27101 raft_consensus.cc:2454] T d0fa289b20bd4f4999803bcf9ba2ed59 P 93791b23b53f4331857f04281bc6b4b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 151ff8be277a442b93c50c965e92a8cc in term 0.
I20260807 05:13:12.270210 26877 leader_election.cc:302] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 151ff8be277a442b93c50c965e92a8cc, 93791b23b53f4331857f04281bc6b4b3; no voters:
I20260807 05:13:12.270202 27247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0fa289b20bd4f4999803bcf9ba2ed59" candidate_uuid: "151ff8be277a442b93c50c965e92a8cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c6243b47d244f5ba25b97daa032b943" is_pre_election: true
I20260807 05:13:12.270462 27247 raft_consensus.cc:2454] T d0fa289b20bd4f4999803bcf9ba2ed59 P 1c6243b47d244f5ba25b97daa032b943 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 151ff8be277a442b93c50c965e92a8cc in term 0.
I20260807 05:13:12.270473 27321 raft_consensus.cc:2781] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:13:12.270545 27321 raft_consensus.cc:482] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:13:12.270586 27321 raft_consensus.cc:3037] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:12.271634 27321 raft_consensus.cc:504] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "151ff8be277a442b93c50c965e92a8cc" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 41691 } } peers { permanent_uuid: "93791b23b53f4331857f04281bc6b4b3" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46333 } } peers { permanent_uuid: "1c6243b47d244f5ba25b97daa032b943" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 45181 } }
I20260807 05:13:12.271945 27321 leader_election.cc:288] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc [CANDIDATE]: Term 1 election: Requested vote from peers 93791b23b53f4331857f04281bc6b4b3 (127.8.16.66:46333), 1c6243b47d244f5ba25b97daa032b943 (127.8.16.67:45181)
I20260807 05:13:12.272241 27247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0fa289b20bd4f4999803bcf9ba2ed59" candidate_uuid: "151ff8be277a442b93c50c965e92a8cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c6243b47d244f5ba25b97daa032b943"
I20260807 05:13:12.272226 27101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0fa289b20bd4f4999803bcf9ba2ed59" candidate_uuid: "151ff8be277a442b93c50c965e92a8cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93791b23b53f4331857f04281bc6b4b3"
I20260807 05:13:12.272334 27101 raft_consensus.cc:3037] T d0fa289b20bd4f4999803bcf9ba2ed59 P 93791b23b53f4331857f04281bc6b4b3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:12.272341 27247 raft_consensus.cc:3037] T d0fa289b20bd4f4999803bcf9ba2ed59 P 1c6243b47d244f5ba25b97daa032b943 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:12.273250 27101 raft_consensus.cc:2454] T d0fa289b20bd4f4999803bcf9ba2ed59 P 93791b23b53f4331857f04281bc6b4b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 151ff8be277a442b93c50c965e92a8cc in term 1.
I20260807 05:13:12.273307 27247 raft_consensus.cc:2454] T d0fa289b20bd4f4999803bcf9ba2ed59 P 1c6243b47d244f5ba25b97daa032b943 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 151ff8be277a442b93c50c965e92a8cc in term 1.
I20260807 05:13:12.273595 26876 leader_election.cc:302] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 151ff8be277a442b93c50c965e92a8cc, 1c6243b47d244f5ba25b97daa032b943; no voters:
I20260807 05:13:12.273761 27321 raft_consensus.cc:2781] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:12.273960 27321 raft_consensus.cc:686] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc [term 1 LEADER]: Becoming Leader. State: Replica: 151ff8be277a442b93c50c965e92a8cc, State: Running, Role: LEADER
I20260807 05:13:12.274240 27321 consensus_queue.cc:237] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "151ff8be277a442b93c50c965e92a8cc" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 41691 } } peers { permanent_uuid: "93791b23b53f4331857f04281bc6b4b3" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46333 } } peers { permanent_uuid: "1c6243b47d244f5ba25b97daa032b943" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 45181 } }
I20260807 05:13:12.277053 26795 catalog_manager.cc:5495] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 151ff8be277a442b93c50c965e92a8cc (127.8.16.65). New cstate: current_term: 1 leader_uuid: "151ff8be277a442b93c50c965e92a8cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "151ff8be277a442b93c50c965e92a8cc" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 41691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93791b23b53f4331857f04281bc6b4b3" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c6243b47d244f5ba25b97daa032b943" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 45181 } health_report { overall_health: UNKNOWN } } }
I20260807 05:13:12.298532 27310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:12.295155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52848 (local address 127.8.16.65:41691)
0807 05:13:12.295268 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:12.295272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:12.295285 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:13:12.295378 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:12.295381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:12.295434 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:12.295513 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:13:12.295688 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.296169 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.297040 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:12.297310 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:12.297392 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:12.297757 (+ 365us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:13:12.298171 (+ 414us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:13:12.298274 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:12.298319 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 05:13:12.298368 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 05:13:12.303987 27101 raft_consensus.cc:1261] T d0fa289b20bd4f4999803bcf9ba2ed59 P 93791b23b53f4331857f04281bc6b4b3 [term 1 FOLLOWER]: Refusing update from remote peer 151ff8be277a442b93c50c965e92a8cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:13:12.303987 27247 raft_consensus.cc:1261] T d0fa289b20bd4f4999803bcf9ba2ed59 P 1c6243b47d244f5ba25b97daa032b943 [term 1 FOLLOWER]: Refusing update from remote peer 151ff8be277a442b93c50c965e92a8cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:13:12.304656 27326 consensus_queue.cc:1035] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "93791b23b53f4331857f04281bc6b4b3" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:13:12.304829 27321 consensus_queue.cc:1035] T d0fa289b20bd4f4999803bcf9ba2ed59 P 151ff8be277a442b93c50c965e92a8cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c6243b47d244f5ba25b97daa032b943" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 45181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:13:12.312124 27331 mvcc.cc:205] Tried to move back new op lower bound from 7315782010070777856 to 7315782009957625856. Current Snapshot: MvccSnapshot[applied={T|T < 7315782010070777856}]
I20260807 05:13:12.319809 26795 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:13:12.319955 26795 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:13:12.322190 26795 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a681e82c7a24e90855e4b4804be535e: Sending DeleteTablet for 3 replicas of tablet d0fa289b20bd4f4999803bcf9ba2ed59
I20260807 05:13:12.322925 26926 tablet_service.cc:1514] Processing DeleteTablet for tablet d0fa289b20bd4f4999803bcf9ba2ed59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:13:12 UTC) from {username='slave'} at 127.0.0.1:52846
I20260807 05:13:12.323011 27073 tablet_service.cc:1514] Processing DeleteTablet for tablet d0fa289b20bd4f4999803bcf9ba2ed59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:13:12 UTC) from {username='slave'} at 127.0.0.1:52524
I20260807 05:13:12.323365 27224 tablet_service.cc:1514] Processing DeleteTablet for tablet d0fa289b20bd4f4999803bcf9ba2ed59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:13:12 UTC) from {username='slave'} at 127.0.0.1:46786
I20260807 05:13:12.323838 27344 tablet_replica.cc:330] T d0fa289b20bd4f4999803bcf9ba2ed59 P 1c6243b47d244f5ba25b97daa032b943: stopping tablet replica
I20260807 05:13:12.324137 27343 tablet_replica.cc:330] T d0fa289b20bd4f4999803bcf9ba2ed59 P 93791b23b53f4331857f04281bc6b4b3: stopping tablet replica
I20260807 05:13:12.324296 27344 raft_consensus.cc:2229] T d0fa289b20bd4f4999803bcf9ba2ed59 P 1c6243b47d244f5ba25b97daa032b943 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:13:12.324473 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 26860
I20260807 05:13:12.325258 27344 raft_consensus.cc:2258] T d0fa289b20bd4f4999803bcf9ba2ed59 P 1c6243b47d244f5ba25b97daa032b943 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:13:12.326988 27343 raft_consensus.cc:2229] T d0fa289b20bd4f4999803bcf9ba2ed59 P 93791b23b53f4331857f04281bc6b4b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:13:12.327044 27344 ts_tablet_manager.cc:1905] T d0fa289b20bd4f4999803bcf9ba2ed59 P 1c6243b47d244f5ba25b97daa032b943: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:13:12.327296 27343 raft_consensus.cc:2258] T d0fa289b20bd4f4999803bcf9ba2ed59 P 93791b23b53f4331857f04281bc6b4b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:13:12.328608 27343 ts_tablet_manager.cc:1905] T d0fa289b20bd4f4999803bcf9ba2ed59 P 93791b23b53f4331857f04281bc6b4b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:13:12.329526 27344 ts_tablet_manager.cc:1918] T d0fa289b20bd4f4999803bcf9ba2ed59 P 1c6243b47d244f5ba25b97daa032b943: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:13:12.329617 27344 log.cc:1196] T d0fa289b20bd4f4999803bcf9ba2ed59 P 1c6243b47d244f5ba25b97daa032b943: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/d0fa289b20bd4f4999803bcf9ba2ed59
I20260807 05:13:12.329880 27344 ts_tablet_manager.cc:1939] T d0fa289b20bd4f4999803bcf9ba2ed59 P 1c6243b47d244f5ba25b97daa032b943: Deleting consensus metadata
I20260807 05:13:12.331053 26777 catalog_manager.cc:4841] TS 1c6243b47d244f5ba25b97daa032b943 (127.8.16.67:45181): tablet d0fa289b20bd4f4999803bcf9ba2ed59 (table test-table [id=5d0afc3e329b4ffab74fc3835452cec4]) successfully deleted
I20260807 05:13:12.331218 27343 ts_tablet_manager.cc:1918] T d0fa289b20bd4f4999803bcf9ba2ed59 P 93791b23b53f4331857f04281bc6b4b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:13:12.331312 27343 log.cc:1196] T d0fa289b20bd4f4999803bcf9ba2ed59 P 93791b23b53f4331857f04281bc6b4b3: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079354071073-8257-0/minicluster-data/ts-1/wal/wals/d0fa289b20bd4f4999803bcf9ba2ed59
I20260807 05:13:12.331593 27343 ts_tablet_manager.cc:1939] T d0fa289b20bd4f4999803bcf9ba2ed59 P 93791b23b53f4331857f04281bc6b4b3: Deleting consensus metadata
I20260807 05:13:12.332733 26778 catalog_manager.cc:4841] TS 93791b23b53f4331857f04281bc6b4b3 (127.8.16.66:46333): tablet d0fa289b20bd4f4999803bcf9ba2ed59 (table test-table [id=5d0afc3e329b4ffab74fc3835452cec4]) successfully deleted
W20260807 05:13:12.333128 26780 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.16.65:41691 (error 108)
I20260807 05:13:12.333336 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 27005
I20260807 05:13:12.336218 27312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:12.335933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:41691 (local address 127.0.0.1:52864)
0807 05:13:12.336066 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:12.336137 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.65:41691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 05:13:12.336370 26780 catalog_manager.cc:4584] TS 151ff8be277a442b93c50c965e92a8cc (127.8.16.65:41691): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d0fa289b20bd4f4999803bcf9ba2ed59: Network error: Client connection negotiation failed: client connection to 127.8.16.65:41691: connect: Connection refused (error 111)
I20260807 05:13:12.340145 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 27157
I20260807 05:13:12.345580 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 26760
2026-08-07T05:13:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1046 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:13:12 dist-test-slave-49h3 krb5kdc[27350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:13:12 dist-test-slave-49h3 krb5kdc[27350](info): set up 2 sockets
Aug 07 05:13:12 dist-test-slave-49h3 krb5kdc[27350](info): commencing operation
krb5kdc: starting...
W20260807 05:13:13.320981 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.954s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:13:13 dist-test-slave-49h3 krb5kdc[27350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:13:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:13:13.438192 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:41439
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:38437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:41439
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:13.546840 27364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:13.547081 27364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:13.547119 27364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:13.550992 27364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:13.551049 27364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:13.551067 27364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:13.551082 27364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:13.551096 27364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:13.556290 27364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:38437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:41439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:41439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:13.557327 27364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:13.558320 27364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:13.566315 27373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:13.566334 27369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:13.566323 27371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:13.566612 27364 server_base.cc:1034] running on GCE node
I20260807 05:13:13.567142 27364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:13.568133 27364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:13.569283 27364 hybrid_clock.cc:648] HybridClock initialized: now 1786079593569265 us; error 37 us; skew 500 ppm
Aug 07 05:13:13 dist-test-slave-49h3 krb5kdc[27350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079593, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:13.571813 27364 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:13:13.572660 27364 webserver.cc:460] Webserver started at http://127.8.16.126:38645/ using document root <none> and password file <none>
I20260807 05:13:13.573177 27364 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:13.573243 27364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:13.573479 27364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:13.575464 27364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "4de3cf356dc54359bba5ed469a7808ed"
format_stamp: "Formatted at 2026-08-07 05:13:13 on dist-test-slave-49h3"
server_key: "ea153b07845f6532d46e1e3eae9e8b9e"
server_key_iv: "22b2e2b3af980212d07b4653bd6e3ed0"
server_key_version: "encryptionkey@0"
I20260807 05:13:13.575954 27364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "4de3cf356dc54359bba5ed469a7808ed"
format_stamp: "Formatted at 2026-08-07 05:13:13 on dist-test-slave-49h3"
server_key: "ea153b07845f6532d46e1e3eae9e8b9e"
server_key_iv: "22b2e2b3af980212d07b4653bd6e3ed0"
server_key_version: "encryptionkey@0"
I20260807 05:13:13.579428 27364 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:13:13.581696 27381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:13.582839 27364 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:13:13.582935 27364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "4de3cf356dc54359bba5ed469a7808ed"
format_stamp: "Formatted at 2026-08-07 05:13:13 on dist-test-slave-49h3"
server_key: "ea153b07845f6532d46e1e3eae9e8b9e"
server_key_iv: "22b2e2b3af980212d07b4653bd6e3ed0"
server_key_version: "encryptionkey@0"
I20260807 05:13:13.583020 27364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:13.610334 27364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:13.613149 27364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:13.613308 27364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:13.619999 27364 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:41439
I20260807 05:13:13.620019 27437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:41439 every 8 connection(s)
I20260807 05:13:13.620913 27364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:13.623550 27438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:13.624409 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 27364
I20260807 05:13:13.624511 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:13.624790 8257 external_mini_cluster.cc:1428] Setting key c03f112dae754f18fe44341484b4a1b4
I20260807 05:13:13.629689 27438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed: Bootstrap starting.
I20260807 05:13:13.632251 27438 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:13.633134 27438 log.cc:824] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed: Log is configured to *not* fsync() on all Append() calls
Aug 07 05:13:13 dist-test-slave-49h3 krb5kdc[27350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:13:13.635020 27438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed: No bootstrap required, opened a new log
I20260807 05:13:13.638170 27441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:13.626077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47804 (local address 127.8.16.126:41439)
0807 05:13:13.626480 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:13.626495 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:13.626529 (+ 34us) server_negotiation.cc:407] Connection header received
0807 05:13:13.627143 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:13.627158 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:13.627478 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:13.627747 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:13.628627 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:13.629643 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:13.630771 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:13.631219 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:13.631327 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:13.634030 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:13.634035 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:13.634047 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:13.634070 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:13.635613 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:13.636053 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:13.636057 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:13.636061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:13.636160 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:13.636397 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:13.636399 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:13.636400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:13.636764 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:13.636906 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:13.637186 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 05:13:13.637433 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":105,"threads_started":1}
I20260807 05:13:13.638797 27438 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de3cf356dc54359bba5ed469a7808ed" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41439 } }
I20260807 05:13:13.639008 27438 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:13.639065 27438 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4de3cf356dc54359bba5ed469a7808ed, State: Initialized, Role: FOLLOWER
I20260807 05:13:13.639500 27438 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de3cf356dc54359bba5ed469a7808ed" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41439 } }
I20260807 05:13:13.639652 27438 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:13.639705 27438 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:13.639784 27438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:13.640683 27438 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de3cf356dc54359bba5ed469a7808ed" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41439 } }
I20260807 05:13:13.640973 27438 leader_election.cc:302] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4de3cf356dc54359bba5ed469a7808ed; no voters:
I20260807 05:13:13.641237 27438 leader_election.cc:288] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:13.641368 27443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:13.641834 27443 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed [term 1 LEADER]: Becoming Leader. State: Replica: 4de3cf356dc54359bba5ed469a7808ed, State: Running, Role: LEADER
I20260807 05:13:13.642127 27443 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de3cf356dc54359bba5ed469a7808ed" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41439 } }
I20260807 05:13:13.642382 27438 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:13.643649 27444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4de3cf356dc54359bba5ed469a7808ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de3cf356dc54359bba5ed469a7808ed" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41439 } } }
I20260807 05:13:13.643675 27445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4de3cf356dc54359bba5ed469a7808ed. Latest consensus state: current_term: 1 leader_uuid: "4de3cf356dc54359bba5ed469a7808ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de3cf356dc54359bba5ed469a7808ed" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41439 } } }
I20260807 05:13:13.643781 27444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:13.643781 27445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:13.644328 27455 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:13.647862 27455 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:13.652535 27455 catalog_manager.cc:1347] Generated new cluster ID: 49e21506672f463383fcd4ccad9cddc5
I20260807 05:13:13.652595 27455 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:13.657903 27455 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:13.658715 27455 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:13.667748 27455 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed: Generated new TSK 0
I20260807 05:13:13.668392 27455 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
I20260807 05:13:13.699365 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:41439
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:38437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:13.808411 27464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:13.808689 27464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:13.808739 27464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:13.812821 27464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:13.812901 27464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:13.812959 27464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:13:13.818321 27464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:38437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:41439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:13.819422 27464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:13.820451 27464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:13.829928 27469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:13.829955 27471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:13.829962 27473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:13.830269 27464 server_base.cc:1034] running on GCE node
I20260807 05:13:13.830751 27464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:13.831382 27464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:13.832568 27464 hybrid_clock.cc:648] HybridClock initialized: now 1786079593832543 us; error 39 us; skew 500 ppm
Aug 07 05:13:13 dist-test-slave-49h3 krb5kdc[27350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079593, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:13.835537 27464 init.cc:377] Logged in from keytab as kudu/127.8.16.65@KRBTEST.COM (short username kudu)
I20260807 05:13:13.836437 27464 webserver.cc:460] Webserver started at http://127.8.16.65:40945/ using document root <none> and password file <none>
I20260807 05:13:13.836948 27464 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:13.837003 27464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:13.837204 27464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:13.839239 27464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "46376b93d6044659ae5bd8cb6309e8a5"
format_stamp: "Formatted at 2026-08-07 05:13:13 on dist-test-slave-49h3"
server_key: "6d191e5073322faa6ed414f53646bb41"
server_key_iv: "9514340d7a1d2db2d23ae03ec7008142"
server_key_version: "encryptionkey@0"
I20260807 05:13:13.839733 27464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "46376b93d6044659ae5bd8cb6309e8a5"
format_stamp: "Formatted at 2026-08-07 05:13:13 on dist-test-slave-49h3"
server_key: "6d191e5073322faa6ed414f53646bb41"
server_key_iv: "9514340d7a1d2db2d23ae03ec7008142"
server_key_version: "encryptionkey@0"
I20260807 05:13:13.843528 27464 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 05:13:13.846047 27481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:13.847289 27464 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:13:13.847393 27464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "46376b93d6044659ae5bd8cb6309e8a5"
format_stamp: "Formatted at 2026-08-07 05:13:13 on dist-test-slave-49h3"
server_key: "6d191e5073322faa6ed414f53646bb41"
server_key_iv: "9514340d7a1d2db2d23ae03ec7008142"
server_key_version: "encryptionkey@0"
I20260807 05:13:13.847503 27464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:13.864995 27464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:13.868145 27464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:13.868330 27464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:13.868911 27464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:13:13.869761 27464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:13:13.869823 27464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:13.869877 27464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:13:13.869902 27464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:13.878729 27464 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:34029
I20260807 05:13:13.878756 27601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:34029 every 8 connection(s)
I20260807 05:13:13.879628 27464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:13:13.885929 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 27464
I20260807 05:13:13.886099 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:13:13.886451 8257 external_mini_cluster.cc:1428] Setting key 4733347a5918058044fe3edf1c6c916b
Aug 07 05:13:13 dist-test-slave-49h3 krb5kdc[27350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079593, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:13:13.893801 27441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:13.881513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:52721 (local address 127.8.16.126:41439)
0807 05:13:13.881651 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:13.881655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:13.882492 (+ 837us) server_negotiation.cc:407] Connection header received
0807 05:13:13.883713 (+ 1221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:13.883716 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:13.883769 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:13.883853 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:13.885012 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:13.885637 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:13.887314 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:13.887600 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:13.887732 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:13.890314 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:13.890319 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:13.890321 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:13.890350 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:13.891635 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:13.892182 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:13.892185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:13.892186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:13.892255 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:13.892621 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:13.892623 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:13.892624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:13.892854 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:13.892936 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:13.893559 (+ 623us) server_negotiation.cc:299] Negotiation successful
0807 05:13:13.893661 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":47}
I20260807 05:13:13.894979 27604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:13.881847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41439 (local address 127.8.16.65:52721)
0807 05:13:13.882330 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:13.882375 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:13.883499 (+ 1124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:13.884015 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:13.884022 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:13.884393 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:13.884871 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:13.884885 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:13.885837 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:13.885840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:13.887186 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:13.887201 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:13.887777 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:13.887781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:13.887936 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:13.888525 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:13.888544 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:13.890145 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:13.891757 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:13.891763 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:13.891779 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:13.892102 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:13.892333 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:13.892336 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:13.892338 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:13.892528 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:13.892958 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:13.892965 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:13.893290 (+ 325us) client_negotiation.cc:769] Sending connection context
0807 05:13:13.893535 (+ 245us) client_negotiation.cc:240] Negotiation successful
0807 05:13:13.893901 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":117,"threads_started":1}
I20260807 05:13:13.896234 27602 heartbeater.cc:347] Connected to a master server at 127.8.16.126:41439
I20260807 05:13:13.896461 27602 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.8.16.66@KRBTEST.COM; defaulting to no policy
I20260807 05:13:13.896936 27602 heartbeater.cc:511] Master 127.8.16.126:41439 requested a full tablet report, sending...
I20260807 05:13:13.898401 27396 ts_manager.cc:194] Registered new tserver with Master: 46376b93d6044659ae5bd8cb6309e8a5 (127.8.16.65:34029)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.66@KRBTEST.COM" created.
I20260807 05:13:13.899597 27396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:52721
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
I20260807 05:13:13.913311 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:41439
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:38437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:14.026249 27607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:14.026501 27607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:14.026561 27607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:14.030572 27607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:14.030637 27607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:14.030717 27607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:13:14.035949 27607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:38437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:41439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:14.036945 27607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:14.037987 27607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:14.047056 27614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:14.047091 27613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:14.047067 27620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:14.047345 27607 server_base.cc:1034] running on GCE node
I20260807 05:13:14.047720 27607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:14.048346 27607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:14.049513 27607 hybrid_clock.cc:648] HybridClock initialized: now 1786079594049496 us; error 28 us; skew 500 ppm
Aug 07 05:13:14 dist-test-slave-49h3 krb5kdc[27350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079594, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:14.055874 27607 init.cc:377] Logged in from keytab as kudu/127.8.16.66@KRBTEST.COM (short username kudu)
I20260807 05:13:14.056779 27607 webserver.cc:460] Webserver started at http://127.8.16.66:33179/ using document root <none> and password file <none>
I20260807 05:13:14.057307 27607 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:14.057363 27607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:14.057580 27607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:14.059562 27607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "e72039d5ec9d465ba488d7ec7c3b3600"
format_stamp: "Formatted at 2026-08-07 05:13:14 on dist-test-slave-49h3"
server_key: "55a3d50daa586fe51ea95f97d14183b1"
server_key_iv: "3cd9c78bb3b18cfb792962839b820819"
server_key_version: "encryptionkey@0"
I20260807 05:13:14.060066 27607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "e72039d5ec9d465ba488d7ec7c3b3600"
format_stamp: "Formatted at 2026-08-07 05:13:14 on dist-test-slave-49h3"
server_key: "55a3d50daa586fe51ea95f97d14183b1"
server_key_iv: "3cd9c78bb3b18cfb792962839b820819"
server_key_version: "encryptionkey@0"
I20260807 05:13:14.063611 27607 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:13:14.065903 27628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:14.067122 27607 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:13:14.067233 27607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "e72039d5ec9d465ba488d7ec7c3b3600"
format_stamp: "Formatted at 2026-08-07 05:13:14 on dist-test-slave-49h3"
server_key: "55a3d50daa586fe51ea95f97d14183b1"
server_key_iv: "3cd9c78bb3b18cfb792962839b820819"
server_key_version: "encryptionkey@0"
I20260807 05:13:14.067320 27607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:14.079650 27607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:14.082775 27607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:14.082932 27607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:14.083475 27607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:13:14.084252 27607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:13:14.084297 27607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:14.084335 27607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:13:14.084355 27607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:14.092764 27607 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:35213
I20260807 05:13:14.092792 27757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:35213 every 8 connection(s)
I20260807 05:13:14.093654 27607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:13:14.095836 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 27607
I20260807 05:13:14.095952 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:13:14.096247 8257 external_mini_cluster.cc:1428] Setting key 7f89ff27807245cf348375bdfb6ba99b
Aug 07 05:13:14 dist-test-slave-49h3 krb5kdc[27350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079594, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.16.67@KRBTEST.COM; defaulting to no policy
I20260807 05:13:14.107216 27441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:14.095568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:44881 (local address 127.8.16.126:41439)
0807 05:13:14.095722 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:14.095727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:14.096660 (+ 933us) server_negotiation.cc:407] Connection header received
0807 05:13:14.097984 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:14.097987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:14.098041 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:14.098122 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:14.099282 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.099806 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.100922 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.101150 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.101237 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:14.103815 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:14.103819 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:14.103821 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:14.103844 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:14.105182 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:14.105723 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:14.105726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:14.105728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:14.105805 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:14.106119 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:14.106122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:14.106124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:14.106317 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:14.106402 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:14.106918 (+ 516us) server_negotiation.cc:299] Negotiation successful
0807 05:13:14.107045 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.67@KRBTEST.COM" created.
I20260807 05:13:14.108021 27760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:14.096005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41439 (local address 127.8.16.66:44881)
0807 05:13:14.096492 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:14.096544 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:14.097775 (+ 1231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:14.098293 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:14.098300 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:14.098667 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:14.099140 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:14.099153 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.099923 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.099926 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:14.100799 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:14.100808 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.101290 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.101294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:14.101479 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:14.102036 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:14.102052 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:14.103646 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:14.105303 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:14.105309 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:14.105325 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:14.105633 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:14.105886 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:14.105889 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:14.105891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:14.106026 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:14.106415 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:14.106422 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:14.106672 (+ 250us) client_negotiation.cc:769] Sending connection context
0807 05:13:14.106838 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 05:13:14.107160 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":110,"threads_started":1}
I20260807 05:13:14.109108 27758 heartbeater.cc:347] Connected to a master server at 127.8.16.126:41439
I20260807 05:13:14.109345 27758 heartbeater.cc:464] Registering TS with master...
I20260807 05:13:14.109856 27758 heartbeater.cc:511] Master 127.8.16.126:41439 requested a full tablet report, sending...
I20260807 05:13:14.110850 27396 ts_manager.cc:194] Registered new tserver with Master: e72039d5ec9d465ba488d7ec7c3b3600 (127.8.16.66:35213)
I20260807 05:13:14.111408 27396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:44881
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
I20260807 05:13:14.121948 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:41439
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:38437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:14.233058 27763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:14.233342 27763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:14.233386 27763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:14.237340 27763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:14.237419 27763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:14.237521 27763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:13:14.242776 27763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:38437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:41439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:14.243825 27763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:14.244831 27763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:14.254280 27772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:14.254329 27768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:14.254329 27770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:14.254627 27763 server_base.cc:1034] running on GCE node
I20260807 05:13:14.255040 27763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:14.255700 27763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:14.256857 27763 hybrid_clock.cc:648] HybridClock initialized: now 1786079594256834 us; error 37 us; skew 500 ppm
Aug 07 05:13:14 dist-test-slave-49h3 krb5kdc[27350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079594, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:14.259636 27763 init.cc:377] Logged in from keytab as kudu/127.8.16.67@KRBTEST.COM (short username kudu)
I20260807 05:13:14.260516 27763 webserver.cc:460] Webserver started at http://127.8.16.67:42521/ using document root <none> and password file <none>
I20260807 05:13:14.261050 27763 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:14.261116 27763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:14.261317 27763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:14.263408 27763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "9f9f5d18f0994be583dafa28fe6e0214"
format_stamp: "Formatted at 2026-08-07 05:13:14 on dist-test-slave-49h3"
server_key: "04abca408fdfd3df21dace83cced7732"
server_key_iv: "1269bd2634ee58a694672a8e8c2b7e5b"
server_key_version: "encryptionkey@0"
I20260807 05:13:14.263952 27763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "9f9f5d18f0994be583dafa28fe6e0214"
format_stamp: "Formatted at 2026-08-07 05:13:14 on dist-test-slave-49h3"
server_key: "04abca408fdfd3df21dace83cced7732"
server_key_iv: "1269bd2634ee58a694672a8e8c2b7e5b"
server_key_version: "encryptionkey@0"
I20260807 05:13:14.267650 27763 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:13:14.269966 27781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:14.271162 27763 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:13:14.271274 27763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "9f9f5d18f0994be583dafa28fe6e0214"
format_stamp: "Formatted at 2026-08-07 05:13:14 on dist-test-slave-49h3"
server_key: "04abca408fdfd3df21dace83cced7732"
server_key_iv: "1269bd2634ee58a694672a8e8c2b7e5b"
server_key_version: "encryptionkey@0"
I20260807 05:13:14.271370 27763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:14.298308 27763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:14.301553 27763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:14.301743 27763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:14.302341 27763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:13:14.303154 27763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:13:14.303201 27763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:14.303263 27763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:13:14.303292 27763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:14.311630 27763 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:38517
I20260807 05:13:14.311658 27906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:38517 every 8 connection(s)
I20260807 05:13:14.312525 27763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:13:14.318809 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 27763
I20260807 05:13:14.318920 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:13:14.319160 8257 external_mini_cluster.cc:1428] Setting key 2e81e06aa5f5f9f50bf0e4a9e6c75d18
Aug 07 05:13:14 dist-test-slave-49h3 krb5kdc[27350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079594, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:13:14.325414 27441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:14.314324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:41371 (local address 127.8.16.126:41439)
0807 05:13:14.314475 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:14.314479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:14.315286 (+ 807us) server_negotiation.cc:407] Connection header received
0807 05:13:14.316503 (+ 1217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:14.316506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:14.316556 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:14.316632 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:14.317867 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.318399 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.319587 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.319846 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.319951 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:14.322539 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:14.322542 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:14.322544 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:14.322569 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:14.323830 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:14.324218 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:14.324220 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:14.324222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:14.324295 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:14.324534 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:14.324536 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:14.324537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:14.324714 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:14.324794 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:14.325196 (+ 402us) server_negotiation.cc:299] Negotiation successful
0807 05:13:14.325292 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 05:13:14.326340 27910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:14.314682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:41439 (local address 127.8.16.67:41371)
0807 05:13:14.315129 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:14.315173 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:14.316308 (+ 1135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:14.316796 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:14.316803 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:14.317173 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:14.317727 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:14.317743 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.318487 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.318489 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:14.319464 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:14.319474 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.320016 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.320020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:14.320226 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:14.320861 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:14.320878 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:14.322388 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:14.323909 (+ 1521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:14.323913 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:14.323924 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:14.324151 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:14.324354 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:14.324356 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:14.324357 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:14.324474 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:14.324792 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:14.324797 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:14.325018 (+ 221us) client_negotiation.cc:769] Sending connection context
0807 05:13:14.325165 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 05:13:14.325416 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":110,"threads_started":1}
I20260807 05:13:14.327374 27908 heartbeater.cc:347] Connected to a master server at 127.8.16.126:41439
I20260807 05:13:14.327584 27908 heartbeater.cc:464] Registering TS with master...
I20260807 05:13:14.328042 27908 heartbeater.cc:511] Master 127.8.16.126:41439 requested a full tablet report, sending...
I20260807 05:13:14.328944 27396 ts_manager.cc:194] Registered new tserver with Master: 9f9f5d18f0994be583dafa28fe6e0214 (127.8.16.67:38517)
I20260807 05:13:14.329528 27396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:41371
I20260807 05:13:14.332688 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:13:14.341605 27441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:14.334583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47818 (local address 127.8.16.126:41439)
0807 05:13:14.334697 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:14.334702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:14.334854 (+ 152us) server_negotiation.cc:407] Connection header received
0807 05:13:14.334969 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:14.334973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:14.335016 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:14.335080 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:14.335774 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.336284 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.337079 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.337307 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.337390 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:14.338846 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:14.338849 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:14.338852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:14.338877 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:14.339890 (+ 1013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:14.340354 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:14.340358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:14.340360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:14.340437 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:14.340737 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:14.340741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:14.340743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:14.340945 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:14.341077 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:14.341332 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 05:13:14.341421 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 05:13:14.344532 27396 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:13:14.347545 27396 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:13:14.361076 27926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:14.356225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34296 (local address 127.8.16.66:35213)
0807 05:13:14.356504 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:14.356516 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:14.356538 (+ 22us) server_negotiation.cc:407] Connection header received
0807 05:13:14.356617 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:14.356621 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:14.356842 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:14.357322 (+ 480us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:14.357498 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.358300 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.359740 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.360451 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.360545 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:14.360689 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:14.360846 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 05:13:14.360940 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":89,"threads_started":1}
I20260807 05:13:14.361069 27925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:14.355544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:35213 (local address 127.0.0.1:34296)
0807 05:13:14.356380 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:14.356392 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:14.356477 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:14.357010 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:14.357016 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:14.357037 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:14.357176 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:14.357183 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.358418 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.358421 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:14.359596 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:14.359606 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.360560 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.360563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:14.360667 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:14.360703 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 05:13:14.360806 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 05:13:14.360920 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"thread_start_us":93,"threads_started":1}
I20260807 05:13:14.361513 27923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:14.356150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:34029 (local address 127.0.0.1:50034)
0807 05:13:14.356547 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:14.356559 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:14.356638 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:14.357504 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:14.357507 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:14.357523 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:14.357600 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:14.357605 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.358887 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.358891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:14.359923 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:14.359932 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.361077 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.361081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:14.361193 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:14.361220 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 05:13:14.361290 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 05:13:14.361352 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":6,"thread_start_us":109,"threads_started":1}
I20260807 05:13:14.361605 27927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:14.356640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50034 (local address 127.8.16.65:34029)
0807 05:13:14.357046 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:14.357059 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:14.357078 (+ 19us) server_negotiation.cc:407] Connection header received
0807 05:13:14.357138 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:14.357142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:14.357287 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:14.357425 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:14.357706 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.358776 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.360035 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.360788 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.360888 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:14.361015 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:14.361341 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 05:13:14.361435 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":120,"threads_started":1}
I20260807 05:13:14.362068 27921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:14.355449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:38517 (local address 127.0.0.1:42908)
0807 05:13:14.355874 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:14.355919 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:14.356035 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:14.357288 (+ 1253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:14.357292 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:14.357308 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:14.357401 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:14.357407 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.358852 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.358856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:14.360070 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:14.360080 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.361084 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.361087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:14.361546 (+ 459us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:14.361824 (+ 278us) client_negotiation.cc:769] Sending connection context
0807 05:13:14.361878 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 05:13:14.361946 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":445,"thread_start_us":100,"threads_started":1}
I20260807 05:13:14.363938 27532 tablet_service.cc:1467] Processing CreateTablet for tablet c228e29819374800b5c237380d49d555 (DEFAULT_TABLE table=test-table [id=c8f918c1be3744b097dc78a942eeeb7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:13:14.363974 27682 tablet_service.cc:1467] Processing CreateTablet for tablet c228e29819374800b5c237380d49d555 (DEFAULT_TABLE table=test-table [id=c8f918c1be3744b097dc78a942eeeb7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:13:14.364847 27682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c228e29819374800b5c237380d49d555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:14.364847 27532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c228e29819374800b5c237380d49d555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:14.370561 27928 tablet_bootstrap.cc:492] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5: Bootstrap starting.
I20260807 05:13:14.370743 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:14.355575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42908 (local address 127.8.16.67:38517)
0807 05:13:14.356861 (+ 1286us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:14.356872 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:14.356893 (+ 21us) server_negotiation.cc:407] Connection header received
0807 05:13:14.356963 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:14.356968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:14.357137 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:14.357306 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:14.357563 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.358713 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.360185 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.360968 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.370167 (+ 9199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:14.370359 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:14.370416 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 05:13:14.370515 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1191,"thread_start_us":96,"threads_started":1}
I20260807 05:13:14.371353 27929 tablet_bootstrap.cc:492] T c228e29819374800b5c237380d49d555 P e72039d5ec9d465ba488d7ec7c3b3600: Bootstrap starting.
I20260807 05:13:14.373107 27928 tablet_bootstrap.cc:654] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:14.373620 27929 tablet_bootstrap.cc:654] T c228e29819374800b5c237380d49d555 P e72039d5ec9d465ba488d7ec7c3b3600: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:14.373839 27834 tablet_service.cc:1467] Processing CreateTablet for tablet c228e29819374800b5c237380d49d555 (DEFAULT_TABLE table=test-table [id=c8f918c1be3744b097dc78a942eeeb7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:13:14.374361 27928 log.cc:824] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:14.374356 27929 log.cc:824] T c228e29819374800b5c237380d49d555 P e72039d5ec9d465ba488d7ec7c3b3600: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:14.374782 27834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c228e29819374800b5c237380d49d555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:14.376339 27929 tablet_bootstrap.cc:492] T c228e29819374800b5c237380d49d555 P e72039d5ec9d465ba488d7ec7c3b3600: No bootstrap required, opened a new log
I20260807 05:13:14.376339 27928 tablet_bootstrap.cc:492] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5: No bootstrap required, opened a new log
I20260807 05:13:14.376523 27929 ts_tablet_manager.cc:1397] T c228e29819374800b5c237380d49d555 P e72039d5ec9d465ba488d7ec7c3b3600: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:13:14.376523 27928 ts_tablet_manager.cc:1397] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260807 05:13:14.380275 27928 raft_consensus.cc:348] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9f5d18f0994be583dafa28fe6e0214" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38517 } } peers { permanent_uuid: "e72039d5ec9d465ba488d7ec7c3b3600" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 35213 } } peers { permanent_uuid: "46376b93d6044659ae5bd8cb6309e8a5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34029 } }
I20260807 05:13:14.380345 27929 raft_consensus.cc:348] T c228e29819374800b5c237380d49d555 P e72039d5ec9d465ba488d7ec7c3b3600 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9f5d18f0994be583dafa28fe6e0214" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38517 } } peers { permanent_uuid: "e72039d5ec9d465ba488d7ec7c3b3600" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 35213 } } peers { permanent_uuid: "46376b93d6044659ae5bd8cb6309e8a5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34029 } }
I20260807 05:13:14.380551 27929 raft_consensus.cc:374] T c228e29819374800b5c237380d49d555 P e72039d5ec9d465ba488d7ec7c3b3600 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:14.380553 27928 raft_consensus.cc:374] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:14.380606 27929 raft_consensus.cc:729] T c228e29819374800b5c237380d49d555 P e72039d5ec9d465ba488d7ec7c3b3600 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72039d5ec9d465ba488d7ec7c3b3600, State: Initialized, Role: FOLLOWER
I20260807 05:13:14.380606 27928 raft_consensus.cc:729] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46376b93d6044659ae5bd8cb6309e8a5, State: Initialized, Role: FOLLOWER
I20260807 05:13:14.380769 27932 tablet_bootstrap.cc:492] T c228e29819374800b5c237380d49d555 P 9f9f5d18f0994be583dafa28fe6e0214: Bootstrap starting.
I20260807 05:13:14.381080 27929 consensus_queue.cc:260] T c228e29819374800b5c237380d49d555 P e72039d5ec9d465ba488d7ec7c3b3600 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9f5d18f0994be583dafa28fe6e0214" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38517 } } peers { permanent_uuid: "e72039d5ec9d465ba488d7ec7c3b3600" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 35213 } } peers { permanent_uuid: "46376b93d6044659ae5bd8cb6309e8a5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34029 } }
I20260807 05:13:14.381080 27928 consensus_queue.cc:260] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9f5d18f0994be583dafa28fe6e0214" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38517 } } peers { permanent_uuid: "e72039d5ec9d465ba488d7ec7c3b3600" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 35213 } } peers { permanent_uuid: "46376b93d6044659ae5bd8cb6309e8a5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34029 } }
I20260807 05:13:14.381762 27602 heartbeater.cc:503] Master 127.8.16.126:41439 was elected leader, sending a full tablet report...
I20260807 05:13:14.382037 27758 heartbeater.cc:503] Master 127.8.16.126:41439 was elected leader, sending a full tablet report...
I20260807 05:13:14.383080 27932 tablet_bootstrap.cc:654] T c228e29819374800b5c237380d49d555 P 9f9f5d18f0994be583dafa28fe6e0214: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:14.383152 27928 ts_tablet_manager.cc:1428] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5: Time spent starting tablet: real 0.007s user 0.004s sys 0.002s
I20260807 05:13:14.383893 27929 ts_tablet_manager.cc:1428] T c228e29819374800b5c237380d49d555 P e72039d5ec9d465ba488d7ec7c3b3600: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 05:13:14.384529 27932 log.cc:824] T c228e29819374800b5c237380d49d555 P 9f9f5d18f0994be583dafa28fe6e0214: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:14.386723 27932 tablet_bootstrap.cc:492] T c228e29819374800b5c237380d49d555 P 9f9f5d18f0994be583dafa28fe6e0214: No bootstrap required, opened a new log
I20260807 05:13:14.386919 27932 ts_tablet_manager.cc:1397] T c228e29819374800b5c237380d49d555 P 9f9f5d18f0994be583dafa28fe6e0214: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 05:13:14.389652 27932 raft_consensus.cc:348] T c228e29819374800b5c237380d49d555 P 9f9f5d18f0994be583dafa28fe6e0214 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9f5d18f0994be583dafa28fe6e0214" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38517 } } peers { permanent_uuid: "e72039d5ec9d465ba488d7ec7c3b3600" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 35213 } } peers { permanent_uuid: "46376b93d6044659ae5bd8cb6309e8a5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34029 } }
I20260807 05:13:14.389873 27932 raft_consensus.cc:374] T c228e29819374800b5c237380d49d555 P 9f9f5d18f0994be583dafa28fe6e0214 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:14.389914 27932 raft_consensus.cc:729] T c228e29819374800b5c237380d49d555 P 9f9f5d18f0994be583dafa28fe6e0214 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f9f5d18f0994be583dafa28fe6e0214, State: Initialized, Role: FOLLOWER
I20260807 05:13:14.390259 27932 consensus_queue.cc:260] T c228e29819374800b5c237380d49d555 P 9f9f5d18f0994be583dafa28fe6e0214 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9f5d18f0994be583dafa28fe6e0214" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38517 } } peers { permanent_uuid: "e72039d5ec9d465ba488d7ec7c3b3600" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 35213 } } peers { permanent_uuid: "46376b93d6044659ae5bd8cb6309e8a5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34029 } }
I20260807 05:13:14.390820 27908 heartbeater.cc:503] Master 127.8.16.126:41439 was elected leader, sending a full tablet report...
I20260807 05:13:14.391032 27932 ts_tablet_manager.cc:1428] T c228e29819374800b5c237380d49d555 P 9f9f5d18f0994be583dafa28fe6e0214: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260807 05:13:14.396167 27935 raft_consensus.cc:482] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:13:14.396330 27935 raft_consensus.cc:504] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9f5d18f0994be583dafa28fe6e0214" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38517 } } peers { permanent_uuid: "e72039d5ec9d465ba488d7ec7c3b3600" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 35213 } } peers { permanent_uuid: "46376b93d6044659ae5bd8cb6309e8a5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34029 } }
I20260807 05:13:14.397284 27935 leader_election.cc:288] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f9f5d18f0994be583dafa28fe6e0214 (127.8.16.67:38517), e72039d5ec9d465ba488d7ec7c3b3600 (127.8.16.66:35213)
I20260807 05:13:14.400985 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:14.397565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:48461 (local address 127.8.16.67:38517)
0807 05:13:14.397685 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:14.397689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:14.397883 (+ 194us) server_negotiation.cc:407] Connection header received
0807 05:13:14.397968 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:14.397971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:14.398021 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:14.398090 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:14.398365 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.398904 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.400064 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.400458 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.400551 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:14.400608 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:14.400792 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 05:13:14.400861 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 05:13:14.401000 27926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:14.397631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:58933 (local address 127.8.16.66:35213)
0807 05:13:14.397804 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:14.397808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:14.398053 (+ 245us) server_negotiation.cc:407] Connection header received
0807 05:13:14.398151 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:14.398154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:14.398197 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:14.398271 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:14.398502 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.398984 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.400065 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.400455 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.400542 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:14.400602 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:14.400823 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 05:13:14.400892 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 05:13:14.401021 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:14.397482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:38517 (local address 127.8.16.65:48461)
0807 05:13:14.397796 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:14.397808 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:14.397897 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:14.398112 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:14.398115 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:14.398132 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:14.398268 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:14.398275 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.399002 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.399006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:14.399949 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:14.399958 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.400565 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.400569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:14.400676 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:14.400701 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 05:13:14.400760 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 05:13:14.400823 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":57,"thread_start_us":99,"threads_started":1}
I20260807 05:13:14.401027 27940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:14.397528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:35213 (local address 127.8.16.65:58933)
0807 05:13:14.397963 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:14.397976 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:14.398076 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:14.398287 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:14.398289 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:14.398303 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:14.398411 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:14.398417 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.399087 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.399090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:14.399950 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:14.399959 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.400570 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.400573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:14.400710 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:14.400728 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 05:13:14.400798 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 05:13:14.400861 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":47,"thread_start_us":89,"threads_started":1}
I20260807 05:13:14.401610 27859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c228e29819374800b5c237380d49d555" candidate_uuid: "46376b93d6044659ae5bd8cb6309e8a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f9f5d18f0994be583dafa28fe6e0214" is_pre_election: true
I20260807 05:13:14.401713 27704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c228e29819374800b5c237380d49d555" candidate_uuid: "46376b93d6044659ae5bd8cb6309e8a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72039d5ec9d465ba488d7ec7c3b3600" is_pre_election: true
I20260807 05:13:14.401909 27859 raft_consensus.cc:2454] T c228e29819374800b5c237380d49d555 P 9f9f5d18f0994be583dafa28fe6e0214 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46376b93d6044659ae5bd8cb6309e8a5 in term 0.
I20260807 05:13:14.401933 27704 raft_consensus.cc:2454] T c228e29819374800b5c237380d49d555 P e72039d5ec9d465ba488d7ec7c3b3600 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46376b93d6044659ae5bd8cb6309e8a5 in term 0.
I20260807 05:13:14.402381 27484 leader_election.cc:302] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46376b93d6044659ae5bd8cb6309e8a5, e72039d5ec9d465ba488d7ec7c3b3600; no voters:
I20260807 05:13:14.402627 27935 raft_consensus.cc:2781] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:13:14.402705 27935 raft_consensus.cc:482] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:13:14.402750 27935 raft_consensus.cc:3037] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:14.403806 27935 raft_consensus.cc:504] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9f5d18f0994be583dafa28fe6e0214" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38517 } } peers { permanent_uuid: "e72039d5ec9d465ba488d7ec7c3b3600" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 35213 } } peers { permanent_uuid: "46376b93d6044659ae5bd8cb6309e8a5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34029 } }
I20260807 05:13:14.404103 27935 leader_election.cc:288] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5 [CANDIDATE]: Term 1 election: Requested vote from peers 9f9f5d18f0994be583dafa28fe6e0214 (127.8.16.67:38517), e72039d5ec9d465ba488d7ec7c3b3600 (127.8.16.66:35213)
I20260807 05:13:14.404422 27704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c228e29819374800b5c237380d49d555" candidate_uuid: "46376b93d6044659ae5bd8cb6309e8a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72039d5ec9d465ba488d7ec7c3b3600"
I20260807 05:13:14.404425 27859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c228e29819374800b5c237380d49d555" candidate_uuid: "46376b93d6044659ae5bd8cb6309e8a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f9f5d18f0994be583dafa28fe6e0214"
I20260807 05:13:14.404548 27859 raft_consensus.cc:3037] T c228e29819374800b5c237380d49d555 P 9f9f5d18f0994be583dafa28fe6e0214 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:14.404548 27704 raft_consensus.cc:3037] T c228e29819374800b5c237380d49d555 P e72039d5ec9d465ba488d7ec7c3b3600 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:14.405810 27859 raft_consensus.cc:2454] T c228e29819374800b5c237380d49d555 P 9f9f5d18f0994be583dafa28fe6e0214 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46376b93d6044659ae5bd8cb6309e8a5 in term 1.
I20260807 05:13:14.405810 27704 raft_consensus.cc:2454] T c228e29819374800b5c237380d49d555 P e72039d5ec9d465ba488d7ec7c3b3600 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46376b93d6044659ae5bd8cb6309e8a5 in term 1.
I20260807 05:13:14.406134 27483 leader_election.cc:302] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 46376b93d6044659ae5bd8cb6309e8a5, 9f9f5d18f0994be583dafa28fe6e0214, e72039d5ec9d465ba488d7ec7c3b3600; no voters:
I20260807 05:13:14.406298 27935 raft_consensus.cc:2781] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:14.406510 27935 raft_consensus.cc:686] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5 [term 1 LEADER]: Becoming Leader. State: Replica: 46376b93d6044659ae5bd8cb6309e8a5, State: Running, Role: LEADER
I20260807 05:13:14.406826 27935 consensus_queue.cc:237] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9f5d18f0994be583dafa28fe6e0214" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38517 } } peers { permanent_uuid: "e72039d5ec9d465ba488d7ec7c3b3600" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 35213 } } peers { permanent_uuid: "46376b93d6044659ae5bd8cb6309e8a5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34029 } }
I20260807 05:13:14.409667 27394 catalog_manager.cc:5495] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46376b93d6044659ae5bd8cb6309e8a5 (127.8.16.65). New cstate: current_term: 1 leader_uuid: "46376b93d6044659ae5bd8cb6309e8a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f9f5d18f0994be583dafa28fe6e0214" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e72039d5ec9d465ba488d7ec7c3b3600" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 35213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46376b93d6044659ae5bd8cb6309e8a5" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 34029 } health_report { overall_health: HEALTHY } } }
I20260807 05:13:14.423749 27927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:14.420385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50044 (local address 127.8.16.65:34029)
0807 05:13:14.420502 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:14.420507 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:14.420520 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:13:14.420676 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:14.420678 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:14.420732 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:14.420818 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:13:14.421050 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.421588 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.422402 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.422606 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.422689 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:14.422986 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:13:14.423366 (+ 380us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:13:14.423492 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:14.423592 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 05:13:14.423628 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 05:13:14.429136 27704 raft_consensus.cc:1261] T c228e29819374800b5c237380d49d555 P e72039d5ec9d465ba488d7ec7c3b3600 [term 1 FOLLOWER]: Refusing update from remote peer 46376b93d6044659ae5bd8cb6309e8a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:13:14.429134 27859 raft_consensus.cc:1261] T c228e29819374800b5c237380d49d555 P 9f9f5d18f0994be583dafa28fe6e0214 [term 1 FOLLOWER]: Refusing update from remote peer 46376b93d6044659ae5bd8cb6309e8a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:13:14.429804 27941 consensus_queue.cc:1035] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72039d5ec9d465ba488d7ec7c3b3600" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 35213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:13:14.429970 27935 consensus_queue.cc:1035] T c228e29819374800b5c237380d49d555 P 46376b93d6044659ae5bd8cb6309e8a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f9f5d18f0994be583dafa28fe6e0214" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 38517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:13:14.436429 27944 mvcc.cc:205] Tried to move back new op lower bound from 7315782018775224320 to 7315782018692792320. Current Snapshot: MvccSnapshot[applied={T|T < 7315782018775224320}]
I20260807 05:13:14.445797 27396 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:13:14.445943 27396 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:13:14.448130 27396 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4de3cf356dc54359bba5ed469a7808ed: Sending DeleteTablet for 3 replicas of tablet c228e29819374800b5c237380d49d555
I20260807 05:13:14.448901 27834 tablet_service.cc:1514] Processing DeleteTablet for tablet c228e29819374800b5c237380d49d555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:13:14 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:42908
I20260807 05:13:14.448971 27682 tablet_service.cc:1514] Processing DeleteTablet for tablet c228e29819374800b5c237380d49d555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:13:14 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:34296
I20260807 05:13:14.449395 27956 tablet_replica.cc:330] T c228e29819374800b5c237380d49d555 P 9f9f5d18f0994be583dafa28fe6e0214: stopping tablet replica
I20260807 05:13:14.449577 27532 tablet_service.cc:1514] Processing DeleteTablet for tablet c228e29819374800b5c237380d49d555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:13:14 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:50034
I20260807 05:13:14.449681 27957 tablet_replica.cc:330] T c228e29819374800b5c237380d49d555 P e72039d5ec9d465ba488d7ec7c3b3600: stopping tablet replica
I20260807 05:13:14.449913 27956 raft_consensus.cc:2229] T c228e29819374800b5c237380d49d555 P 9f9f5d18f0994be583dafa28fe6e0214 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:13:14.449952 27957 raft_consensus.cc:2229] T c228e29819374800b5c237380d49d555 P e72039d5ec9d465ba488d7ec7c3b3600 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:13:14.450081 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 27464
I20260807 05:13:14.450248 27957 raft_consensus.cc:2258] T c228e29819374800b5c237380d49d555 P e72039d5ec9d465ba488d7ec7c3b3600 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:13:14.450976 27956 raft_consensus.cc:2258] T c228e29819374800b5c237380d49d555 P 9f9f5d18f0994be583dafa28fe6e0214 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:13:14.452241 27956 ts_tablet_manager.cc:1905] T c228e29819374800b5c237380d49d555 P 9f9f5d18f0994be583dafa28fe6e0214: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:13:14.452301 27957 ts_tablet_manager.cc:1905] T c228e29819374800b5c237380d49d555 P e72039d5ec9d465ba488d7ec7c3b3600: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:13:14.454416 27956 ts_tablet_manager.cc:1918] T c228e29819374800b5c237380d49d555 P 9f9f5d18f0994be583dafa28fe6e0214: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:13:14.454516 27956 log.cc:1196] T c228e29819374800b5c237380d49d555 P 9f9f5d18f0994be583dafa28fe6e0214: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/c228e29819374800b5c237380d49d555
I20260807 05:13:14.454800 27956 ts_tablet_manager.cc:1939] T c228e29819374800b5c237380d49d555 P 9f9f5d18f0994be583dafa28fe6e0214: Deleting consensus metadata
I20260807 05:13:14.455328 27957 ts_tablet_manager.cc:1918] T c228e29819374800b5c237380d49d555 P e72039d5ec9d465ba488d7ec7c3b3600: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:13:14.455444 27957 log.cc:1196] T c228e29819374800b5c237380d49d555 P e72039d5ec9d465ba488d7ec7c3b3600: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079354071073-8257-0/minicluster-data/ts-1/wal/wals/c228e29819374800b5c237380d49d555
I20260807 05:13:14.455817 27957 ts_tablet_manager.cc:1939] T c228e29819374800b5c237380d49d555 P e72039d5ec9d465ba488d7ec7c3b3600: Deleting consensus metadata
I20260807 05:13:14.455935 27383 catalog_manager.cc:4841] TS 9f9f5d18f0994be583dafa28fe6e0214 (127.8.16.67:38517): tablet c228e29819374800b5c237380d49d555 (table test-table [id=c8f918c1be3744b097dc78a942eeeb7f]) successfully deleted
I20260807 05:13:14.457073 27384 catalog_manager.cc:4841] TS e72039d5ec9d465ba488d7ec7c3b3600 (127.8.16.66:35213): tablet c228e29819374800b5c237380d49d555 (table test-table [id=c8f918c1be3744b097dc78a942eeeb7f]) successfully deleted
W20260807 05:13:14.461380 27382 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.16.65:34029 (error 108)
I20260807 05:13:14.461692 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 27607
I20260807 05:13:14.464412 27923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:14.464136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:34029 (local address 127.0.0.1:50052)
0807 05:13:14.464275 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:14.464339 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.65:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 05:13:14.464581 27382 catalog_manager.cc:4584] TS 46376b93d6044659ae5bd8cb6309e8a5 (127.8.16.65:34029): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c228e29819374800b5c237380d49d555: Network error: Client connection negotiation failed: client connection to 127.8.16.65:34029: connect: Connection refused (error 111)
I20260807 05:13:14.467337 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 27763
I20260807 05:13:14.472400 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 27364
2026-08-07T05:13:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2126 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T05:13:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:14Z Disabled control of system clock
I20260807 05:13:14.501108 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:36475
--webserver_interface=127.8.16.126
--webserver_port=0
--builtin_ntp_servers=127.8.16.84:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:36475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 05:13:14.609851 27964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:14.610126 27964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:14.610162 27964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:14.614099 27964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:14.614166 27964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:14.614179 27964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:14.614195 27964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:14.614208 27964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:14.619005 27964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:36475
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:36475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:14.620025 27964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:14.621358 27964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:14.629426 27972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:14.629416 27969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:14.629420 27970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:14.629724 27964 server_base.cc:1034] running on GCE node
I20260807 05:13:14.630326 27964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:14.631379 27964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:14.632531 27964 hybrid_clock.cc:648] HybridClock initialized: now 1786079594632514 us; error 30 us; skew 500 ppm
I20260807 05:13:14.634352 27964 webserver.cc:460] Webserver started at http://127.8.16.126:45317/ using document root <none> and password file <none>
I20260807 05:13:14.634897 27964 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:14.634960 27964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:14.635159 27964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:14.637179 27964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "5d037923dc874ef39a3ca55b7ceab5f2"
format_stamp: "Formatted at 2026-08-07 05:13:14 on dist-test-slave-49h3"
server_key: "6cc04c673a4a336bd39f31641402c5ba"
server_key_iv: "243ac76d07df1e1d99495e9be07549be"
server_key_version: "encryptionkey@0"
I20260807 05:13:14.637748 27964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "5d037923dc874ef39a3ca55b7ceab5f2"
format_stamp: "Formatted at 2026-08-07 05:13:14 on dist-test-slave-49h3"
server_key: "6cc04c673a4a336bd39f31641402c5ba"
server_key_iv: "243ac76d07df1e1d99495e9be07549be"
server_key_version: "encryptionkey@0"
I20260807 05:13:14.641155 27964 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:13:14.643425 27979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:14.644577 27964 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:13:14.644685 27964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "5d037923dc874ef39a3ca55b7ceab5f2"
format_stamp: "Formatted at 2026-08-07 05:13:14 on dist-test-slave-49h3"
server_key: "6cc04c673a4a336bd39f31641402c5ba"
server_key_iv: "243ac76d07df1e1d99495e9be07549be"
server_key_version: "encryptionkey@0"
I20260807 05:13:14.644781 27964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:14.666988 27964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:14.667701 27964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:14.667891 27964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:14.674472 27964 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:36475
I20260807 05:13:14.674484 28034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:36475 every 8 connection(s)
I20260807 05:13:14.675396 27964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:14.676756 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 27964
I20260807 05:13:14.676873 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:14.677132 8257 external_mini_cluster.cc:1428] Setting key 46ea664d10601941f9b51b4e3e28ef90
I20260807 05:13:14.678674 28035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:14.683969 28035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2: Bootstrap starting.
I20260807 05:13:14.685847 28038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:14.678441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58706 (local address 127.8.16.126:36475)
0807 05:13:14.679081 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:14.679097 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:14.679130 (+ 33us) server_negotiation.cc:407] Connection header received
0807 05:13:14.679763 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:14.679778 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:14.680140 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:14.680438 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:14.680854 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.681884 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.682748 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.683199 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.683349 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:14.684150 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:14.684157 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:14.684172 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:14.684554 (+ 382us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:14.684611 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:14.684632 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:14.684715 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:14.684859 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 05:13:14.685103 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":158,"threads_started":1}
I20260807 05:13:14.686228 28035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:14.687059 28035 log.cc:824] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:14.688886 28035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2: No bootstrap required, opened a new log
I20260807 05:13:14.692168 28035 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d037923dc874ef39a3ca55b7ceab5f2" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36475 } }
I20260807 05:13:14.692368 28035 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:14.692420 28035 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d037923dc874ef39a3ca55b7ceab5f2, State: Initialized, Role: FOLLOWER
I20260807 05:13:14.692808 28035 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d037923dc874ef39a3ca55b7ceab5f2" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36475 } }
I20260807 05:13:14.692934 28035 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:14.692993 28035 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:14.693077 28035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:14.694041 28035 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d037923dc874ef39a3ca55b7ceab5f2" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36475 } }
I20260807 05:13:14.694334 28035 leader_election.cc:302] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d037923dc874ef39a3ca55b7ceab5f2; no voters:
I20260807 05:13:14.694604 28035 leader_election.cc:288] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:14.694700 28040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:14.694900 28040 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d037923dc874ef39a3ca55b7ceab5f2, State: Running, Role: LEADER
I20260807 05:13:14.695168 28040 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d037923dc874ef39a3ca55b7ceab5f2" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36475 } }
I20260807 05:13:14.695648 28035 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:14.696683 28041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d037923dc874ef39a3ca55b7ceab5f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d037923dc874ef39a3ca55b7ceab5f2" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36475 } } }
I20260807 05:13:14.696683 28042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d037923dc874ef39a3ca55b7ceab5f2. Latest consensus state: current_term: 1 leader_uuid: "5d037923dc874ef39a3ca55b7ceab5f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d037923dc874ef39a3ca55b7ceab5f2" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36475 } } }
I20260807 05:13:14.696913 28041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:14.696930 28042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:14.697952 28052 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:14.700347 28052 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:14.705318 28052 catalog_manager.cc:1347] Generated new cluster ID: 182e4d0be9df4966b950fb24b5e9730b
I20260807 05:13:14.705379 28052 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:14.716050 28052 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:14.717274 28052 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:14.726076 28052 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2: Generated new TSK 0
I20260807 05:13:14.726958 28052 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:13:14.735172 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36475
--builtin_ntp_servers=127.8.16.84:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 05:13:14.844434 28061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:14.844720 28061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:14.844779 28061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:14.848872 28061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:14.848927 28061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:14.849000 28061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:13:14.853786 28061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:14.854828 28061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:14.856165 28061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:14.865619 28066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:14.865674 28067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:14.865682 28069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:14.865913 28061 server_base.cc:1034] running on GCE node
I20260807 05:13:14.866379 28061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:14.866971 28061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:14.868120 28061 hybrid_clock.cc:648] HybridClock initialized: now 1786079594868097 us; error 39 us; skew 500 ppm
I20260807 05:13:14.869886 28061 webserver.cc:460] Webserver started at http://127.8.16.65:45851/ using document root <none> and password file <none>
I20260807 05:13:14.870421 28061 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:14.870489 28061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:14.870668 28061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:14.872625 28061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "0c04e5a62bb349ea9fa5671c82632272"
format_stamp: "Formatted at 2026-08-07 05:13:14 on dist-test-slave-49h3"
server_key: "edf12ebedf9d99a6c8cc609ca2240e5e"
server_key_iv: "ec4e452c420c4f83f343bf8c7761c3bb"
server_key_version: "encryptionkey@0"
I20260807 05:13:14.873147 28061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "0c04e5a62bb349ea9fa5671c82632272"
format_stamp: "Formatted at 2026-08-07 05:13:14 on dist-test-slave-49h3"
server_key: "edf12ebedf9d99a6c8cc609ca2240e5e"
server_key_iv: "ec4e452c420c4f83f343bf8c7761c3bb"
server_key_version: "encryptionkey@0"
I20260807 05:13:14.876518 28061 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260807 05:13:14.878803 28076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:14.879880 28061 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:13:14.879987 28061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "0c04e5a62bb349ea9fa5671c82632272"
format_stamp: "Formatted at 2026-08-07 05:13:14 on dist-test-slave-49h3"
server_key: "edf12ebedf9d99a6c8cc609ca2240e5e"
server_key_iv: "ec4e452c420c4f83f343bf8c7761c3bb"
server_key_version: "encryptionkey@0"
I20260807 05:13:14.880079 28061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:14.894531 28061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:14.895169 28061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:14.895346 28061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:14.895885 28061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:13:14.896808 28061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:13:14.896857 28061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:14.896906 28061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:13:14.896934 28061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:14.904908 28061 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:44831
I20260807 05:13:14.904938 28196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:44831 every 8 connection(s)
I20260807 05:13:14.905753 28061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:13:14.911814 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 28061
I20260807 05:13:14.911921 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:13:14.912175 8257 external_mini_cluster.cc:1428] Setting key c7db0494f5b7b38ce2e64ab6880e2474
I20260807 05:13:14.914175 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36475
--builtin_ntp_servers=127.8.16.84:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 05:13:14.914332 28038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:14.907413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:45667 (local address 127.8.16.126:36475)
0807 05:13:14.907545 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:14.907549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:14.908508 (+ 959us) server_negotiation.cc:407] Connection header received
0807 05:13:14.909838 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:14.909840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:14.909891 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:14.909961 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:14.910738 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.911258 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.912391 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.912667 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.912779 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:14.913881 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:14.913885 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:14.913888 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:14.913901 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:14.913925 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:14.913935 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:14.914011 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:14.914114 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 05:13:14.914184 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 05:13:14.915091 28199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:14.907779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36475 (local address 127.8.16.65:45667)
0807 05:13:14.908381 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:14.908417 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:14.909663 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:14.910096 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:14.910101 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:14.910181 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:14.910631 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:14.910642 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.911344 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.911347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:14.912280 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:14.912289 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:14.912817 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:14.912820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:14.912999 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:14.913634 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:13:14.913637 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:14.913678 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:13:14.913744 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:14.914008 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:14.914012 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:14.914016 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 05:13:14.914094 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 05:13:14.914328 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":116,"threads_started":1}
I20260807 05:13:14.916188 28197 heartbeater.cc:347] Connected to a master server at 127.8.16.126:36475
I20260807 05:13:14.916412 28197 heartbeater.cc:464] Registering TS with master...
I20260807 05:13:14.916860 28197 heartbeater.cc:511] Master 127.8.16.126:36475 requested a full tablet report, sending...
I20260807 05:13:14.918195 27998 ts_manager.cc:194] Registered new tserver with Master: 0c04e5a62bb349ea9fa5671c82632272 (127.8.16.65:44831)
I20260807 05:13:14.919401 27998 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.16.65:45667
W20260807 05:13:15.026336 28201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:15.026671 28201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:15.026724 28201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:15.030804 28201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:15.030859 28201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:15.030926 28201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:13:15.035676 28201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:15.036680 28201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:15.038067 28201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:15.047032 28207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:15.047042 28209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:15.047037 28206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:15.047461 28201 server_base.cc:1034] running on GCE node
I20260807 05:13:15.047849 28201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:15.048481 28201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:15.049623 28201 hybrid_clock.cc:648] HybridClock initialized: now 1786079595049597 us; error 39 us; skew 500 ppm
I20260807 05:13:15.051256 28201 webserver.cc:460] Webserver started at http://127.8.16.66:33919/ using document root <none> and password file <none>
I20260807 05:13:15.051777 28201 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:15.051839 28201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:15.052032 28201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:15.054050 28201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "0668f2bd6cc441818b19c525aa57f339"
format_stamp: "Formatted at 2026-08-07 05:13:15 on dist-test-slave-49h3"
server_key: "bd48c28486e60bafba50c7f5876b710c"
server_key_iv: "9281e8ff76ddb8b37eecb18d01316434"
server_key_version: "encryptionkey@0"
I20260807 05:13:15.054565 28201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "0668f2bd6cc441818b19c525aa57f339"
format_stamp: "Formatted at 2026-08-07 05:13:15 on dist-test-slave-49h3"
server_key: "bd48c28486e60bafba50c7f5876b710c"
server_key_iv: "9281e8ff76ddb8b37eecb18d01316434"
server_key_version: "encryptionkey@0"
I20260807 05:13:15.057961 28201 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:13:15.060118 28215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:15.061233 28201 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:13:15.061335 28201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "0668f2bd6cc441818b19c525aa57f339"
format_stamp: "Formatted at 2026-08-07 05:13:15 on dist-test-slave-49h3"
server_key: "bd48c28486e60bafba50c7f5876b710c"
server_key_iv: "9281e8ff76ddb8b37eecb18d01316434"
server_key_version: "encryptionkey@0"
I20260807 05:13:15.061424 28201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:15.078982 28201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:15.079649 28201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:15.079826 28201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:15.080389 28201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:13:15.081321 28201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:13:15.081368 28201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:15.081419 28201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:13:15.081501 28201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:15.089736 28201 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:46105
I20260807 05:13:15.089768 28341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:46105 every 8 connection(s)
I20260807 05:13:15.090591 28201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:13:15.090965 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 28201
I20260807 05:13:15.091092 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:13:15.091377 8257 external_mini_cluster.cc:1428] Setting key 9762e8aeaccc2185907aeddfad415b26
I20260807 05:13:15.093497 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36475
--builtin_ntp_servers=127.8.16.84:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 05:13:15.100662 28038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:15.092336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:42483 (local address 127.8.16.126:36475)
0807 05:13:15.092450 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:15.092453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:15.093716 (+ 1263us) server_negotiation.cc:407] Connection header received
0807 05:13:15.095406 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:15.095409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:15.095465 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:15.095541 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:15.096462 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.097054 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.098729 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.098947 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.099024 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:15.100228 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:15.100231 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:15.100233 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:15.100247 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:15.100267 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:15.100275 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:15.100327 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:15.100488 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 05:13:15.100547 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 05:13:15.101873 28344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:15.092831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36475 (local address 127.8.16.66:42483)
0807 05:13:15.093568 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:15.093623 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:15.095158 (+ 1535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:15.095755 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:15.095762 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:15.095843 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:15.096326 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:15.096338 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.097214 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.097218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:15.098613 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:15.098624 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.099090 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.099094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:15.099306 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:15.099928 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:13:15.099930 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:15.099988 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:13:15.100074 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:15.100350 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:15.100359 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:15.100365 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 05:13:15.100496 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 05:13:15.100803 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":107,"threads_started":1}
I20260807 05:13:15.103104 28342 heartbeater.cc:347] Connected to a master server at 127.8.16.126:36475
I20260807 05:13:15.103331 28342 heartbeater.cc:464] Registering TS with master...
I20260807 05:13:15.103829 28342 heartbeater.cc:511] Master 127.8.16.126:36475 requested a full tablet report, sending...
I20260807 05:13:15.104820 27998 ts_manager.cc:194] Registered new tserver with Master: 0668f2bd6cc441818b19c525aa57f339 (127.8.16.66:46105)
I20260807 05:13:15.105365 27998 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.16.66:42483
W20260807 05:13:15.204288 28345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:15.204551 28345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:15.204605 28345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:15.208596 28345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:15.208660 28345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:15.208735 28345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:13:15.213474 28345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:15.214583 28345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:15.215986 28345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:15.225023 28352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:15.225042 28354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:15.225024 28350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:15.225181 28345 server_base.cc:1034] running on GCE node
I20260807 05:13:15.225711 28345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:15.226392 28345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:15.227547 28345 hybrid_clock.cc:648] HybridClock initialized: now 1786079595227527 us; error 31 us; skew 500 ppm
I20260807 05:13:15.229280 28345 webserver.cc:460] Webserver started at http://127.8.16.67:43531/ using document root <none> and password file <none>
I20260807 05:13:15.229859 28345 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:15.229925 28345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:15.230098 28345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:15.232009 28345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "df45677035e9470e9d03e4724a12bae3"
format_stamp: "Formatted at 2026-08-07 05:13:15 on dist-test-slave-49h3"
server_key: "2a9d2d82fd2079d6150ac6385f06aba2"
server_key_iv: "b00fa099c58abb32da55509269ca88f0"
server_key_version: "encryptionkey@0"
I20260807 05:13:15.232493 28345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "df45677035e9470e9d03e4724a12bae3"
format_stamp: "Formatted at 2026-08-07 05:13:15 on dist-test-slave-49h3"
server_key: "2a9d2d82fd2079d6150ac6385f06aba2"
server_key_iv: "b00fa099c58abb32da55509269ca88f0"
server_key_version: "encryptionkey@0"
I20260807 05:13:15.235945 28345 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:13:15.238196 28361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:15.239336 28345 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:13:15.239434 28345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "df45677035e9470e9d03e4724a12bae3"
format_stamp: "Formatted at 2026-08-07 05:13:15 on dist-test-slave-49h3"
server_key: "2a9d2d82fd2079d6150ac6385f06aba2"
server_key_iv: "b00fa099c58abb32da55509269ca88f0"
server_key_version: "encryptionkey@0"
I20260807 05:13:15.239533 28345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:15.258039 28345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:15.258718 28345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:15.258872 28345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:15.259404 28345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:13:15.260322 28345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:13:15.260370 28345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:15.260406 28345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:13:15.260426 28345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:15.268532 28345 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:43449
I20260807 05:13:15.268579 28483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:43449 every 8 connection(s)
I20260807 05:13:15.269400 28345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:13:15.270536 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 28345
I20260807 05:13:15.270637 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:13:15.270911 8257 external_mini_cluster.cc:1428] Setting key 00b707a8d70a53fc3f20ec12752c8188
I20260807 05:13:15.278272 28038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:15.271205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:49407 (local address 127.8.16.126:36475)
0807 05:13:15.271362 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:15.271367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:15.272514 (+ 1147us) server_negotiation.cc:407] Connection header received
0807 05:13:15.273945 (+ 1431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:15.273948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:15.274003 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:15.274072 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:15.274919 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.275448 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.276534 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.276767 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.276852 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:15.277835 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:15.277838 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:15.277841 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:15.277854 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:15.277875 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:15.277882 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:15.277938 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:15.278067 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 05:13:15.278128 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 05:13:15.279042 28486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:15.271654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36475 (local address 127.8.16.67:49407)
0807 05:13:15.272388 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:15.272429 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:15.273763 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:15.274238 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:15.274244 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:15.274324 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:15.274791 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:15.274803 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.275559 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.275561 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:15.276422 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:15.276431 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.276898 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.276902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:15.277052 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:15.277608 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:13:15.277609 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:15.277646 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:13:15.277704 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:15.277946 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:15.277955 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:15.277961 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 05:13:15.278060 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 05:13:15.278330 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":125,"threads_started":1}
I20260807 05:13:15.279958 28484 heartbeater.cc:347] Connected to a master server at 127.8.16.126:36475
I20260807 05:13:15.280140 28484 heartbeater.cc:464] Registering TS with master...
I20260807 05:13:15.280568 28484 heartbeater.cc:511] Master 127.8.16.126:36475 requested a full tablet report, sending...
I20260807 05:13:15.281509 27998 ts_manager.cc:194] Registered new tserver with Master: df45677035e9470e9d03e4724a12bae3 (127.8.16.67:43449)
I20260807 05:13:15.282058 27998 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.16.67:49407
I20260807 05:13:15.284552 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:13:15.289690 28038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:15.286442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58712 (local address 127.8.16.126:36475)
0807 05:13:15.286571 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:15.286575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:15.286707 (+ 132us) server_negotiation.cc:407] Connection header received
0807 05:13:15.286801 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:15.286802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:15.286838 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:15.286897 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:15.287102 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.287565 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.288298 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.288491 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.288553 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:15.289280 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:15.289283 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:15.289285 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:15.289295 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:15.289314 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:15.289321 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:15.289410 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:15.289532 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 05:13:15.289561 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 05:13:15.292104 27998 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:13:15.295085 27998 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:13:15.308096 28497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:15.302671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:44831 (local address 127.0.0.1:46258)
0807 05:13:15.303395 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:15.303442 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:15.303572 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:15.304454 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:15.304456 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:15.304469 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:15.304544 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:15.304548 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.305502 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.305505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:15.306668 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:15.306679 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.307562 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.307565 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:15.307667 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:15.307706 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 05:13:15.307826 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 05:13:15.307945 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"spinlock_wait_cycles":3712,"thread_start_us":101,"threads_started":1}
I20260807 05:13:15.308118 28498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:15.302880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46258 (local address 127.8.16.65:44831)
0807 05:13:15.303858 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:15.303870 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:15.303893 (+ 23us) server_negotiation.cc:407] Connection header received
0807 05:13:15.303964 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:15.303969 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:15.304153 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:15.304306 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:15.304650 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.305388 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.306813 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.307459 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.307569 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:15.307765 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:15.307861 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 05:13:15.307952 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":877,"thread_start_us":114,"threads_started":1}
I20260807 05:13:15.308383 28496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:15.302569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:46105 (local address 127.0.0.1:36942)
0807 05:13:15.303393 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:15.303443 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:15.303572 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:15.304247 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:15.304253 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:15.304279 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:15.304424 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:15.304432 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.305985 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.305988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:15.307063 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:15.307072 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.308034 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.308038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:15.308117 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:15.308140 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:13:15.308187 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 05:13:15.308262 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"spinlock_wait_cycles":4736,"thread_start_us":80,"threads_started":1}
I20260807 05:13:15.309056 28499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:15.303233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36942 (local address 127.8.16.66:46105)
0807 05:13:15.303769 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:15.303781 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:15.303806 (+ 25us) server_negotiation.cc:407] Connection header received
0807 05:13:15.303882 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:15.303887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:15.304058 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:15.304573 (+ 515us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:15.304713 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.305863 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.307165 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.307916 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.308620 (+ 704us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:15.308773 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:15.308818 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 05:13:15.308903 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":141,"threads_started":1}
I20260807 05:13:15.309182 28501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:15.303353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49868 (local address 127.8.16.67:43449)
0807 05:13:15.303858 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:15.303870 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:15.304277 (+ 407us) server_negotiation.cc:407] Connection header received
0807 05:13:15.304483 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:15.304489 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:15.304651 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:15.304799 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:15.305156 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.306200 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.307502 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.308627 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.308729 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:15.308890 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:15.308942 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 05:13:15.309028 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":123,"threads_started":1}
I20260807 05:13:15.309816 28495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:15.302924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:43449 (local address 127.0.0.1:49868)
0807 05:13:15.304181 (+ 1257us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:15.304193 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:15.304349 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:15.304939 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:15.304941 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:15.304959 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:15.305038 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:15.305043 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.306308 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.306310 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:15.307374 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:15.307384 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.308752 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.308755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:15.308839 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:15.308860 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 05:13:15.308917 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 05:13:15.309005 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1202,"thread_start_us":77,"threads_started":1}
I20260807 05:13:15.311548 28126 tablet_service.cc:1467] Processing CreateTablet for tablet ad6303a8645843959554569d380dbb0d (DEFAULT_TABLE table=test-table [id=a0183f3e8bed48c0b3b0eb42f3ea0012]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:13:15.311599 28270 tablet_service.cc:1467] Processing CreateTablet for tablet ad6303a8645843959554569d380dbb0d (DEFAULT_TABLE table=test-table [id=a0183f3e8bed48c0b3b0eb42f3ea0012]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:13:15.312882 28126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad6303a8645843959554569d380dbb0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:15.312971 28270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad6303a8645843959554569d380dbb0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:15.312602 28412 tablet_service.cc:1467] Processing CreateTablet for tablet ad6303a8645843959554569d380dbb0d (DEFAULT_TABLE table=test-table [id=a0183f3e8bed48c0b3b0eb42f3ea0012]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:13:15.313687 28412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad6303a8645843959554569d380dbb0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:15.321271 28503 tablet_bootstrap.cc:492] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3: Bootstrap starting.
I20260807 05:13:15.321664 28502 tablet_bootstrap.cc:492] T ad6303a8645843959554569d380dbb0d P 0668f2bd6cc441818b19c525aa57f339: Bootstrap starting.
I20260807 05:13:15.322259 28504 tablet_bootstrap.cc:492] T ad6303a8645843959554569d380dbb0d P 0c04e5a62bb349ea9fa5671c82632272: Bootstrap starting.
I20260807 05:13:15.323784 28503 tablet_bootstrap.cc:654] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:15.323784 28502 tablet_bootstrap.cc:654] T ad6303a8645843959554569d380dbb0d P 0668f2bd6cc441818b19c525aa57f339: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:15.323835 28504 tablet_bootstrap.cc:654] T ad6303a8645843959554569d380dbb0d P 0c04e5a62bb349ea9fa5671c82632272: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:15.324536 28504 log.cc:824] T ad6303a8645843959554569d380dbb0d P 0c04e5a62bb349ea9fa5671c82632272: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:15.324601 28502 log.cc:824] T ad6303a8645843959554569d380dbb0d P 0668f2bd6cc441818b19c525aa57f339: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:15.324641 28503 log.cc:824] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:15.326683 28502 tablet_bootstrap.cc:492] T ad6303a8645843959554569d380dbb0d P 0668f2bd6cc441818b19c525aa57f339: No bootstrap required, opened a new log
I20260807 05:13:15.326680 28503 tablet_bootstrap.cc:492] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3: No bootstrap required, opened a new log
I20260807 05:13:15.326874 28503 ts_tablet_manager.cc:1397] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 05:13:15.326874 28502 ts_tablet_manager.cc:1397] T ad6303a8645843959554569d380dbb0d P 0668f2bd6cc441818b19c525aa57f339: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:13:15.327440 28504 tablet_bootstrap.cc:492] T ad6303a8645843959554569d380dbb0d P 0c04e5a62bb349ea9fa5671c82632272: No bootstrap required, opened a new log
I20260807 05:13:15.327618 28504 ts_tablet_manager.cc:1397] T ad6303a8645843959554569d380dbb0d P 0c04e5a62bb349ea9fa5671c82632272: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 05:13:15.330202 28502 raft_consensus.cc:348] T ad6303a8645843959554569d380dbb0d P 0668f2bd6cc441818b19c525aa57f339 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0668f2bd6cc441818b19c525aa57f339" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46105 } } peers { permanent_uuid: "0c04e5a62bb349ea9fa5671c82632272" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 44831 } } peers { permanent_uuid: "df45677035e9470e9d03e4724a12bae3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 43449 } }
I20260807 05:13:15.330437 28502 raft_consensus.cc:374] T ad6303a8645843959554569d380dbb0d P 0668f2bd6cc441818b19c525aa57f339 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:15.330506 28502 raft_consensus.cc:729] T ad6303a8645843959554569d380dbb0d P 0668f2bd6cc441818b19c525aa57f339 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0668f2bd6cc441818b19c525aa57f339, State: Initialized, Role: FOLLOWER
I20260807 05:13:15.330938 28502 consensus_queue.cc:260] T ad6303a8645843959554569d380dbb0d P 0668f2bd6cc441818b19c525aa57f339 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0668f2bd6cc441818b19c525aa57f339" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46105 } } peers { permanent_uuid: "0c04e5a62bb349ea9fa5671c82632272" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 44831 } } peers { permanent_uuid: "df45677035e9470e9d03e4724a12bae3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 43449 } }
I20260807 05:13:15.331084 28503 raft_consensus.cc:348] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0668f2bd6cc441818b19c525aa57f339" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46105 } } peers { permanent_uuid: "0c04e5a62bb349ea9fa5671c82632272" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 44831 } } peers { permanent_uuid: "df45677035e9470e9d03e4724a12bae3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 43449 } }
I20260807 05:13:15.331382 28503 raft_consensus.cc:374] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:15.331468 28503 raft_consensus.cc:729] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df45677035e9470e9d03e4724a12bae3, State: Initialized, Role: FOLLOWER
I20260807 05:13:15.331599 28504 raft_consensus.cc:348] T ad6303a8645843959554569d380dbb0d P 0c04e5a62bb349ea9fa5671c82632272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0668f2bd6cc441818b19c525aa57f339" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46105 } } peers { permanent_uuid: "0c04e5a62bb349ea9fa5671c82632272" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 44831 } } peers { permanent_uuid: "df45677035e9470e9d03e4724a12bae3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 43449 } }
I20260807 05:13:15.331843 28504 raft_consensus.cc:374] T ad6303a8645843959554569d380dbb0d P 0c04e5a62bb349ea9fa5671c82632272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:15.331897 28502 ts_tablet_manager.cc:1428] T ad6303a8645843959554569d380dbb0d P 0668f2bd6cc441818b19c525aa57f339: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 05:13:15.331925 28504 raft_consensus.cc:729] T ad6303a8645843959554569d380dbb0d P 0c04e5a62bb349ea9fa5671c82632272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c04e5a62bb349ea9fa5671c82632272, State: Initialized, Role: FOLLOWER
I20260807 05:13:15.331898 28503 consensus_queue.cc:260] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0668f2bd6cc441818b19c525aa57f339" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46105 } } peers { permanent_uuid: "0c04e5a62bb349ea9fa5671c82632272" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 44831 } } peers { permanent_uuid: "df45677035e9470e9d03e4724a12bae3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 43449 } }
I20260807 05:13:15.332274 28342 heartbeater.cc:503] Master 127.8.16.126:36475 was elected leader, sending a full tablet report...
I20260807 05:13:15.332376 28504 consensus_queue.cc:260] T ad6303a8645843959554569d380dbb0d P 0c04e5a62bb349ea9fa5671c82632272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0668f2bd6cc441818b19c525aa57f339" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46105 } } peers { permanent_uuid: "0c04e5a62bb349ea9fa5671c82632272" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 44831 } } peers { permanent_uuid: "df45677035e9470e9d03e4724a12bae3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 43449 } }
I20260807 05:13:15.332861 28503 ts_tablet_manager.cc:1428] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 05:13:15.333186 28484 heartbeater.cc:503] Master 127.8.16.126:36475 was elected leader, sending a full tablet report...
I20260807 05:13:15.333240 28504 ts_tablet_manager.cc:1428] T ad6303a8645843959554569d380dbb0d P 0c04e5a62bb349ea9fa5671c82632272: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 05:13:15.333554 28197 heartbeater.cc:503] Master 127.8.16.126:36475 was elected leader, sending a full tablet report...
W20260807 05:13:15.341581 28343 tablet.cc:2369] T ad6303a8645843959554569d380dbb0d P 0668f2bd6cc441818b19c525aa57f339: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:13:15.362474 28509 raft_consensus.cc:482] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:13:15.362680 28509 raft_consensus.cc:504] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0668f2bd6cc441818b19c525aa57f339" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46105 } } peers { permanent_uuid: "0c04e5a62bb349ea9fa5671c82632272" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 44831 } } peers { permanent_uuid: "df45677035e9470e9d03e4724a12bae3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 43449 } }
I20260807 05:13:15.363785 28509 leader_election.cc:288] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0668f2bd6cc441818b19c525aa57f339 (127.8.16.66:46105), 0c04e5a62bb349ea9fa5671c82632272 (127.8.16.65:44831)
I20260807 05:13:15.367478 28499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:15.364148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:54101 (local address 127.8.16.66:46105)
0807 05:13:15.364268 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:15.364273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:15.364289 (+ 16us) server_negotiation.cc:407] Connection header received
0807 05:13:15.364396 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:15.364399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:15.364454 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:15.364524 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:15.364828 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.365434 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.366576 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.366971 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.367049 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:15.367112 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:15.367292 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 05:13:15.367361 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 05:13:15.367487 28498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:15.364144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:52725 (local address 127.8.16.65:44831)
0807 05:13:15.364277 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:15.364281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:15.364530 (+ 249us) server_negotiation.cc:407] Connection header received
0807 05:13:15.364633 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:15.364636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:15.364687 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:15.364765 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:15.364998 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.365601 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.366617 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.367001 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.367095 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:15.367154 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:15.367323 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 05:13:15.367389 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 05:13:15.367471 28486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:15.363994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:46105 (local address 127.8.16.67:54101)
0807 05:13:15.364183 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:15.364198 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:15.364288 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:15.364546 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:15.364549 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:15.364565 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:15.364692 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:15.364699 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.365554 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.365557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:15.366465 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:15.366474 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.367079 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.367082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:15.367176 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:15.367200 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 05:13:15.367260 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 05:13:15.367328 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260807 05:13:15.367493 28511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:15.364012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:44831 (local address 127.8.16.67:52725)
0807 05:13:15.364431 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:15.364445 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:15.364543 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:15.364775 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:15.364778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:15.364793 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:15.364900 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:15.364906 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.365701 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.365704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:15.366493 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:15.366502 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.367127 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.367131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:15.367201 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:15.367225 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 05:13:15.367295 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 05:13:15.367357 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":117,"threads_started":1}
I20260807 05:13:15.368163 28292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad6303a8645843959554569d380dbb0d" candidate_uuid: "df45677035e9470e9d03e4724a12bae3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0668f2bd6cc441818b19c525aa57f339" is_pre_election: true
I20260807 05:13:15.368162 28146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad6303a8645843959554569d380dbb0d" candidate_uuid: "df45677035e9470e9d03e4724a12bae3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c04e5a62bb349ea9fa5671c82632272" is_pre_election: true
I20260807 05:13:15.368476 28146 raft_consensus.cc:2454] T ad6303a8645843959554569d380dbb0d P 0c04e5a62bb349ea9fa5671c82632272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df45677035e9470e9d03e4724a12bae3 in term 0.
I20260807 05:13:15.368476 28292 raft_consensus.cc:2454] T ad6303a8645843959554569d380dbb0d P 0668f2bd6cc441818b19c525aa57f339 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df45677035e9470e9d03e4724a12bae3 in term 0.
I20260807 05:13:15.368957 28366 leader_election.cc:302] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c04e5a62bb349ea9fa5671c82632272, df45677035e9470e9d03e4724a12bae3; no voters:
I20260807 05:13:15.369220 28509 raft_consensus.cc:2781] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:13:15.369298 28509 raft_consensus.cc:482] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:13:15.369328 28509 raft_consensus.cc:3037] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:15.370342 28509 raft_consensus.cc:504] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0668f2bd6cc441818b19c525aa57f339" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46105 } } peers { permanent_uuid: "0c04e5a62bb349ea9fa5671c82632272" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 44831 } } peers { permanent_uuid: "df45677035e9470e9d03e4724a12bae3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 43449 } }
I20260807 05:13:15.370658 28509 leader_election.cc:288] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3 [CANDIDATE]: Term 1 election: Requested vote from peers 0668f2bd6cc441818b19c525aa57f339 (127.8.16.66:46105), 0c04e5a62bb349ea9fa5671c82632272 (127.8.16.65:44831)
I20260807 05:13:15.371003 28292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad6303a8645843959554569d380dbb0d" candidate_uuid: "df45677035e9470e9d03e4724a12bae3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0668f2bd6cc441818b19c525aa57f339"
I20260807 05:13:15.371027 28146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad6303a8645843959554569d380dbb0d" candidate_uuid: "df45677035e9470e9d03e4724a12bae3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c04e5a62bb349ea9fa5671c82632272"
I20260807 05:13:15.371134 28292 raft_consensus.cc:3037] T ad6303a8645843959554569d380dbb0d P 0668f2bd6cc441818b19c525aa57f339 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:15.371151 28146 raft_consensus.cc:3037] T ad6303a8645843959554569d380dbb0d P 0c04e5a62bb349ea9fa5671c82632272 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:15.372241 28292 raft_consensus.cc:2454] T ad6303a8645843959554569d380dbb0d P 0668f2bd6cc441818b19c525aa57f339 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df45677035e9470e9d03e4724a12bae3 in term 1.
I20260807 05:13:15.372249 28146 raft_consensus.cc:2454] T ad6303a8645843959554569d380dbb0d P 0c04e5a62bb349ea9fa5671c82632272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df45677035e9470e9d03e4724a12bae3 in term 1.
I20260807 05:13:15.372519 28364 leader_election.cc:302] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0668f2bd6cc441818b19c525aa57f339, df45677035e9470e9d03e4724a12bae3; no voters:
I20260807 05:13:15.372676 28509 raft_consensus.cc:2781] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:15.372872 28509 raft_consensus.cc:686] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3 [term 1 LEADER]: Becoming Leader. State: Replica: df45677035e9470e9d03e4724a12bae3, State: Running, Role: LEADER
I20260807 05:13:15.373207 28509 consensus_queue.cc:237] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0668f2bd6cc441818b19c525aa57f339" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46105 } } peers { permanent_uuid: "0c04e5a62bb349ea9fa5671c82632272" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 44831 } } peers { permanent_uuid: "df45677035e9470e9d03e4724a12bae3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 43449 } }
I20260807 05:13:15.376222 27998 catalog_manager.cc:5495] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3 reported cstate change: term changed from 0 to 1, leader changed from <none> to df45677035e9470e9d03e4724a12bae3 (127.8.16.67). New cstate: current_term: 1 leader_uuid: "df45677035e9470e9d03e4724a12bae3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0668f2bd6cc441818b19c525aa57f339" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c04e5a62bb349ea9fa5671c82632272" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 44831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df45677035e9470e9d03e4724a12bae3" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260807 05:13:15.394974 28501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:15.391553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49870 (local address 127.8.16.67:43449)
0807 05:13:15.391673 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:15.391677 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:15.391692 (+ 15us) server_negotiation.cc:407] Connection header received
0807 05:13:15.391819 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:15.391822 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:15.391881 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:15.391968 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:13:15.392215 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.392703 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.393534 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:15.393759 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:15.393844 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:15.394136 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:13:15.394537 (+ 401us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:13:15.394667 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:15.394788 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 05:13:15.394822 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 05:13:15.400466 28292 raft_consensus.cc:1261] T ad6303a8645843959554569d380dbb0d P 0668f2bd6cc441818b19c525aa57f339 [term 1 FOLLOWER]: Refusing update from remote peer df45677035e9470e9d03e4724a12bae3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:13:15.400466 28146 raft_consensus.cc:1261] T ad6303a8645843959554569d380dbb0d P 0c04e5a62bb349ea9fa5671c82632272 [term 1 FOLLOWER]: Refusing update from remote peer df45677035e9470e9d03e4724a12bae3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:13:15.401113 28512 consensus_queue.cc:1035] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0668f2bd6cc441818b19c525aa57f339" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 46105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:13:15.401283 28509 consensus_queue.cc:1035] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c04e5a62bb349ea9fa5671c82632272" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 44831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 05:13:15.406839 28198 tablet.cc:2369] T ad6303a8645843959554569d380dbb0d P 0c04e5a62bb349ea9fa5671c82632272: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:13:15.416644 27998 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:13:15.416828 27998 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:13:15.419106 27998 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5d037923dc874ef39a3ca55b7ceab5f2: Sending DeleteTablet for 3 replicas of tablet ad6303a8645843959554569d380dbb0d
I20260807 05:13:15.419797 28126 tablet_service.cc:1514] Processing DeleteTablet for tablet ad6303a8645843959554569d380dbb0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:13:15 UTC) from {username='slave'} at 127.0.0.1:46258
I20260807 05:13:15.419898 28412 tablet_service.cc:1514] Processing DeleteTablet for tablet ad6303a8645843959554569d380dbb0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:13:15 UTC) from {username='slave'} at 127.0.0.1:49868
I20260807 05:13:15.420251 28270 tablet_service.cc:1514] Processing DeleteTablet for tablet ad6303a8645843959554569d380dbb0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:13:15 UTC) from {username='slave'} at 127.0.0.1:36942
I20260807 05:13:15.420446 28527 tablet_replica.cc:330] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3: stopping tablet replica
I20260807 05:13:15.420722 28527 raft_consensus.cc:2229] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:13:15.420745 28528 tablet_replica.cc:330] T ad6303a8645843959554569d380dbb0d P 0668f2bd6cc441818b19c525aa57f339: stopping tablet replica
I20260807 05:13:15.420799 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 28061
I20260807 05:13:15.421856 28527 raft_consensus.cc:2258] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:13:15.421970 28528 raft_consensus.cc:2229] T ad6303a8645843959554569d380dbb0d P 0668f2bd6cc441818b19c525aa57f339 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:13:15.422221 28528 raft_consensus.cc:2258] T ad6303a8645843959554569d380dbb0d P 0668f2bd6cc441818b19c525aa57f339 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:13:15.423691 28528 ts_tablet_manager.cc:1905] T ad6303a8645843959554569d380dbb0d P 0668f2bd6cc441818b19c525aa57f339: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:13:15.423707 28527 ts_tablet_manager.cc:1905] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:13:15.426446 28528 ts_tablet_manager.cc:1918] T ad6303a8645843959554569d380dbb0d P 0668f2bd6cc441818b19c525aa57f339: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:13:15.426534 28528 log.cc:1196] T ad6303a8645843959554569d380dbb0d P 0668f2bd6cc441818b19c525aa57f339: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-1/wal/wals/ad6303a8645843959554569d380dbb0d
I20260807 05:13:15.426754 28527 ts_tablet_manager.cc:1918] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:13:15.426852 28527 log.cc:1196] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/ad6303a8645843959554569d380dbb0d
I20260807 05:13:15.426856 28528 ts_tablet_manager.cc:1939] T ad6303a8645843959554569d380dbb0d P 0668f2bd6cc441818b19c525aa57f339: Deleting consensus metadata
I20260807 05:13:15.427209 28527 ts_tablet_manager.cc:1939] T ad6303a8645843959554569d380dbb0d P df45677035e9470e9d03e4724a12bae3: Deleting consensus metadata
I20260807 05:13:15.428087 27981 catalog_manager.cc:4841] TS 0668f2bd6cc441818b19c525aa57f339 (127.8.16.66:46105): tablet ad6303a8645843959554569d380dbb0d (table test-table [id=a0183f3e8bed48c0b3b0eb42f3ea0012]) successfully deleted
I20260807 05:13:15.428390 27980 catalog_manager.cc:4841] TS df45677035e9470e9d03e4724a12bae3 (127.8.16.67:43449): tablet ad6303a8645843959554569d380dbb0d (table test-table [id=a0183f3e8bed48c0b3b0eb42f3ea0012]) successfully deleted
W20260807 05:13:15.439178 27984 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.16.65:44831 (error 108)
I20260807 05:13:15.439363 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 28201
I20260807 05:13:15.441941 28495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:15.441603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:44831 (local address 127.0.0.1:46274)
0807 05:13:15.441766 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:15.441845 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.65:44831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 05:13:15.442143 27984 catalog_manager.cc:4584] TS 0c04e5a62bb349ea9fa5671c82632272 (127.8.16.65:44831): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ad6303a8645843959554569d380dbb0d: Network error: Client connection negotiation failed: client connection to 127.8.16.65:44831: connect: Connection refused (error 111)
I20260807 05:13:15.445376 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 28345
I20260807 05:13:15.451310 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 27964
2026-08-07T05:13:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (979 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:13:15 dist-test-slave-49h3 krb5kdc[28532](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:13:15 dist-test-slave-49h3 krb5kdc[28532](info): set up 2 sockets
Aug 07 05:13:15 dist-test-slave-49h3 krb5kdc[28532](info): commencing operation
krb5kdc: starting...
W20260807 05:13:16.425243 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.953s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:13:16 dist-test-slave-49h3 krb5kdc[28532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:13:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:13:16.544104 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:33311
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:33311
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:16.653182 28547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:16.653498 28547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:16.653561 28547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:16.657565 28547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:16.657618 28547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:16.657635 28547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:16.657651 28547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:16.657675 28547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:16.662947 28547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:33311
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:33311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:16.663928 28547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:16.664883 28547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:16.672852 28555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:16.672930 28553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:16.672852 28552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:16.673065 28547 server_base.cc:1034] running on GCE node
I20260807 05:13:16.673678 28547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:16.674713 28547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:16.675870 28547 hybrid_clock.cc:648] HybridClock initialized: now 1786079596675857 us; error 30 us; skew 500 ppm
Aug 07 05:13:16 dist-test-slave-49h3 krb5kdc[28532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079596, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:16.678453 28547 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:13:16.679313 28547 webserver.cc:460] Webserver started at http://127.8.16.126:36473/ using document root <none> and password file <none>
I20260807 05:13:16.679852 28547 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:16.679917 28547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:16.680124 28547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:16.682188 28547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "dcfe521b3f084ce7a0c5f975979dabd8"
format_stamp: "Formatted at 2026-08-07 05:13:16 on dist-test-slave-49h3"
server_key: "2b403d6a6e077909fc9afaeb5ec472b6"
server_key_iv: "5e3c1d8f643b5e2703442d4968e79544"
server_key_version: "encryptionkey@0"
I20260807 05:13:16.682727 28547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "dcfe521b3f084ce7a0c5f975979dabd8"
format_stamp: "Formatted at 2026-08-07 05:13:16 on dist-test-slave-49h3"
server_key: "2b403d6a6e077909fc9afaeb5ec472b6"
server_key_iv: "5e3c1d8f643b5e2703442d4968e79544"
server_key_version: "encryptionkey@0"
I20260807 05:13:16.686280 28547 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 05:13:16.688503 28562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:16.689688 28547 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:13:16.689801 28547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "dcfe521b3f084ce7a0c5f975979dabd8"
format_stamp: "Formatted at 2026-08-07 05:13:16 on dist-test-slave-49h3"
server_key: "2b403d6a6e077909fc9afaeb5ec472b6"
server_key_iv: "5e3c1d8f643b5e2703442d4968e79544"
server_key_version: "encryptionkey@0"
I20260807 05:13:16.689895 28547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:16.702538 28547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:16.705142 28547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:16.705327 28547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:16.712126 28547 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:33311
I20260807 05:13:16.712164 28618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:33311 every 8 connection(s)
I20260807 05:13:16.713035 28547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:16.715684 28619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:16.720257 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 28547
I20260807 05:13:16.720405 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:16.720710 8257 external_mini_cluster.cc:1428] Setting key 016a1740442d5323d6b0d0c174ee589c
I20260807 05:13:16.721472 28619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8: Bootstrap starting.
I20260807 05:13:16.723892 28619 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:16.724623 28619 log.cc:824] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:16.726691 28619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8: No bootstrap required, opened a new log
Aug 07 05:13:16 dist-test-slave-49h3 krb5kdc[28532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:13:16.730774 28619 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcfe521b3f084ce7a0c5f975979dabd8" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 33311 } }
I20260807 05:13:16.731035 28619 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:16.731096 28619 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcfe521b3f084ce7a0c5f975979dabd8, State: Initialized, Role: FOLLOWER
I20260807 05:13:16.731557 28619 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcfe521b3f084ce7a0c5f975979dabd8" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 33311 } }
I20260807 05:13:16.731705 28619 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:16.731772 28619 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:16.731866 28619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:16.732976 28619 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcfe521b3f084ce7a0c5f975979dabd8" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 33311 } }
I20260807 05:13:16.733301 28619 leader_election.cc:302] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcfe521b3f084ce7a0c5f975979dabd8; no voters:
I20260807 05:13:16.733677 28619 leader_election.cc:288] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:16.733763 28624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:16.734198 28624 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8 [term 1 LEADER]: Becoming Leader. State: Replica: dcfe521b3f084ce7a0c5f975979dabd8, State: Running, Role: LEADER
I20260807 05:13:16.734526 28624 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcfe521b3f084ce7a0c5f975979dabd8" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 33311 } }
I20260807 05:13:16.735011 28619 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:16.735184 28622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:16.722175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56320 (local address 127.8.16.126:33311)
0807 05:13:16.722626 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:16.722641 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:16.722676 (+ 35us) server_negotiation.cc:407] Connection header received
0807 05:13:16.723312 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:16.723327 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:16.723648 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:16.723952 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:16.724768 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:16.725975 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:16.726831 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:16.727297 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:16.727443 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:16.730260 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:16.730266 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:16.730279 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:16.730307 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:16.732039 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:16.732484 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:16.732488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:16.732493 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:16.732600 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:16.732871 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:16.732873 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:16.732874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:16.733292 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:16.733494 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:16.733830 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 05:13:16.734187 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":144,"threads_started":1}
I20260807 05:13:16.735728 28624 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcfe521b3f084ce7a0c5f975979dabd8. Latest consensus state: current_term: 1 leader_uuid: "dcfe521b3f084ce7a0c5f975979dabd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcfe521b3f084ce7a0c5f975979dabd8" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 33311 } } }
I20260807 05:13:16.735917 28624 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:16.736135 28625 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcfe521b3f084ce7a0c5f975979dabd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcfe521b3f084ce7a0c5f975979dabd8" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 33311 } } }
I20260807 05:13:16.736299 28625 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:16.736342 28633 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:16.739391 28633 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:16.744582 28633 catalog_manager.cc:1347] Generated new cluster ID: 8db2e5fd1b844c0783d846fd22508045
I20260807 05:13:16.744638 28633 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:16.766770 28633 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:16.767722 28633 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:16.782352 28633 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8: Generated new TSK 0
I20260807 05:13:16.783236 28633 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
Entry for principal kudu/127.8.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab.
I20260807 05:13:16.818516 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:33311
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:16.928072 28645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:16.928330 28645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:16.928373 28645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:16.932444 28645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:16.932507 28645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:16.932569 28645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:13:16.937953 28645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.65.keytab
--principal=kudu/127.8.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:33311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:16.938951 28645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:16.939931 28645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:16.948930 28651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:16.948933 28650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:16.948948 28653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:16.949255 28645 server_base.cc:1034] running on GCE node
I20260807 05:13:16.949636 28645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:16.950199 28645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:16.951367 28645 hybrid_clock.cc:648] HybridClock initialized: now 1786079596951339 us; error 44 us; skew 500 ppm
Aug 07 05:13:16 dist-test-slave-49h3 krb5kdc[28532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079596, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:16.954092 28645 init.cc:377] Logged in from keytab as kudu/127.8.16.65@KRBTEST.COM (short username kudu)
I20260807 05:13:16.954953 28645 webserver.cc:460] Webserver started at http://127.8.16.65:37917/ using document root <none> and password file <none>
I20260807 05:13:16.955475 28645 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:16.955541 28645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:16.955735 28645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:16.957799 28645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "d348fb0a5ffe4c7bb3fbce39609960f2"
format_stamp: "Formatted at 2026-08-07 05:13:16 on dist-test-slave-49h3"
server_key: "1e3cf1caf1d2b7c7b8f3bc62ef0ec584"
server_key_iv: "e07d5fb57399310578e5bea0acdf54e5"
server_key_version: "encryptionkey@0"
I20260807 05:13:16.958335 28645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "d348fb0a5ffe4c7bb3fbce39609960f2"
format_stamp: "Formatted at 2026-08-07 05:13:16 on dist-test-slave-49h3"
server_key: "1e3cf1caf1d2b7c7b8f3bc62ef0ec584"
server_key_iv: "e07d5fb57399310578e5bea0acdf54e5"
server_key_version: "encryptionkey@0"
I20260807 05:13:16.961876 28645 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:13:16.964128 28661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:16.965356 28645 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:13:16.965488 28645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "d348fb0a5ffe4c7bb3fbce39609960f2"
format_stamp: "Formatted at 2026-08-07 05:13:16 on dist-test-slave-49h3"
server_key: "1e3cf1caf1d2b7c7b8f3bc62ef0ec584"
server_key_iv: "e07d5fb57399310578e5bea0acdf54e5"
server_key_version: "encryptionkey@0"
I20260807 05:13:16.965595 28645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:16.977471 28645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:16.980057 28645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:16.980239 28645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:16.980786 28645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:13:16.981602 28645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:13:16.981652 28645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:16.981704 28645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:13:16.981732 28645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:16.989893 28645 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:46365
I20260807 05:13:16.989916 28780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:46365 every 8 connection(s)
I20260807 05:13:16.990765 28645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:13:16.995285 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 28645
I20260807 05:13:16.995398 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:13:16.995668 8257 external_mini_cluster.cc:1428] Setting key 3416dbe0dbf89ded92d99648c524efae
Aug 07 05:13:16 dist-test-slave-49h3 krb5kdc[28532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079596, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.65@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:13:17.003880 28622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:16.992534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:47357 (local address 127.8.16.126:33311)
0807 05:13:16.992668 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:16.992672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:16.993530 (+ 858us) server_negotiation.cc:407] Connection header received
0807 05:13:16.994821 (+ 1291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:16.994824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:16.994873 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:16.994948 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:16.996189 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:16.996790 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:16.998025 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:16.998263 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:16.998352 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:17.000590 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:17.000593 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:17.000595 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:17.000618 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:17.001993 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:17.002463 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:17.002466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:17.002467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:17.002540 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:17.002841 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:17.002844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:17.002845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:17.003014 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:17.003097 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:17.003576 (+ 479us) server_negotiation.cc:299] Negotiation successful
0807 05:13:17.003714 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
WARNING: no policy specified for kudu/127.8.16.66@KRBTEST.COM; defaulting to no policy
I20260807 05:13:17.004683 28784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:16.992931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:33311 (local address 127.8.16.65:47357)
0807 05:13:16.993372 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:16.993414 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:16.994621 (+ 1207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:16.995089 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:16.995095 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:16.995491 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:16.996009 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:16.996029 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:16.996916 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:16.996919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:16.997906 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:16.997915 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:16.998401 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:16.998404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:16.998562 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:16.999099 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:16.999115 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:17.000441 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:17.002098 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:17.002103 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:17.002118 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:17.002372 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:17.002618 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:17.002621 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:17.002623 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:17.002756 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:17.003109 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:17.003116 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:17.003358 (+ 242us) client_negotiation.cc:769] Sending connection context
0807 05:13:17.003519 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 05:13:17.003836 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":129,"threads_started":1}
I20260807 05:13:17.005851 28781 heartbeater.cc:347] Connected to a master server at 127.8.16.126:33311
I20260807 05:13:17.006057 28781 heartbeater.cc:464] Registering TS with master...
I20260807 05:13:17.006609 28781 heartbeater.cc:511] Master 127.8.16.126:33311 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.66@KRBTEST.COM" created.
I20260807 05:13:17.008028 28582 ts_manager.cc:194] Registered new tserver with Master: d348fb0a5ffe4c7bb3fbce39609960f2 (127.8.16.65:46365)
I20260807 05:13:17.009089 28582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.65@KRBTEST.COM'} at 127.8.16.65:47357
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
Entry for principal kudu/127.8.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab.
I20260807 05:13:17.020103 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:33311
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:17.128415 28787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:17.128701 28787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:17.128746 28787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:17.132939 28787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:17.133010 28787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:17.133072 28787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:13:17.138542 28787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.66.keytab
--principal=kudu/127.8.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:33311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:17.139577 28787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:17.140561 28787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:17.149843 28795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:17.149838 28793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:17.149843 28792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:17.150030 28787 server_base.cc:1034] running on GCE node
I20260807 05:13:17.150584 28787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:17.151192 28787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:17.152359 28787 hybrid_clock.cc:648] HybridClock initialized: now 1786079597152344 us; error 30 us; skew 500 ppm
Aug 07 05:13:17 dist-test-slave-49h3 krb5kdc[28532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079597, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:17.154973 28787 init.cc:377] Logged in from keytab as kudu/127.8.16.66@KRBTEST.COM (short username kudu)
I20260807 05:13:17.155814 28787 webserver.cc:460] Webserver started at http://127.8.16.66:45915/ using document root <none> and password file <none>
I20260807 05:13:17.156338 28787 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:17.156397 28787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:17.156560 28787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:17.158521 28787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "755fd643af2d451e9d4bd64b9ad38da8"
format_stamp: "Formatted at 2026-08-07 05:13:17 on dist-test-slave-49h3"
server_key: "bbd4e141e3608fb06a4d7cbe780b7d0b"
server_key_iv: "db1cb809a70b69afec085b1d30a79344"
server_key_version: "encryptionkey@0"
I20260807 05:13:17.158994 28787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "755fd643af2d451e9d4bd64b9ad38da8"
format_stamp: "Formatted at 2026-08-07 05:13:17 on dist-test-slave-49h3"
server_key: "bbd4e141e3608fb06a4d7cbe780b7d0b"
server_key_iv: "db1cb809a70b69afec085b1d30a79344"
server_key_version: "encryptionkey@0"
I20260807 05:13:17.162393 28787 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:13:17.164537 28802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:17.165686 28787 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:13:17.165796 28787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "755fd643af2d451e9d4bd64b9ad38da8"
format_stamp: "Formatted at 2026-08-07 05:13:17 on dist-test-slave-49h3"
server_key: "bbd4e141e3608fb06a4d7cbe780b7d0b"
server_key_iv: "db1cb809a70b69afec085b1d30a79344"
server_key_version: "encryptionkey@0"
I20260807 05:13:17.165882 28787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:17.175172 28787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:17.177932 28787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:17.178092 28787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:17.178608 28787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:13:17.179345 28787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:13:17.179391 28787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:17.179440 28787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:13:17.179469 28787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:17.187721 28787 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:37739
I20260807 05:13:17.187747 28922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:37739 every 8 connection(s)
I20260807 05:13:17.188530 28787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:13:17.196581 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 28787
I20260807 05:13:17.196709 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:13:17.196962 8257 external_mini_cluster.cc:1428] Setting key 91fecb6bc94aa59a4067569452215721
Aug 07 05:13:17 dist-test-slave-49h3 krb5kdc[28532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079597, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.66@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:13:17.201377 28622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:17.190237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:44777 (local address 127.8.16.126:33311)
0807 05:13:17.190366 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:17.190370 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:17.191099 (+ 729us) server_negotiation.cc:407] Connection header received
0807 05:13:17.192339 (+ 1240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:17.192343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:17.192423 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:17.192503 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:17.193788 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.194354 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.195433 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.195649 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.195720 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:17.198026 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:17.198030 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:17.198033 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:17.198056 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:17.199375 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:17.199903 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:17.199906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:17.199907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:17.199972 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:17.200319 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:17.200323 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:17.200325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:17.200545 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:17.200637 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:17.201076 (+ 439us) server_negotiation.cc:299] Negotiation successful
0807 05:13:17.201220 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 05:13:17.202091 28925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:17.190554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:33311 (local address 127.8.16.66:44777)
0807 05:13:17.190965 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:17.191004 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:17.192142 (+ 1138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:17.192733 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:17.192740 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:17.193104 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:17.193643 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:17.193659 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.194462 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.194465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:17.195338 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:17.195349 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.195755 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.195757 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:17.195889 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:17.196420 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:17.196437 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:17.197882 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:17.199483 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:17.199489 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:17.199502 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:17.199822 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:17.200038 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:17.200041 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:17.200043 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:17.200215 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:17.200643 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:17.200648 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:17.200865 (+ 217us) client_negotiation.cc:769] Sending connection context
0807 05:13:17.201009 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 05:13:17.201254 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":115,"threads_started":1}
I20260807 05:13:17.203167 28923 heartbeater.cc:347] Connected to a master server at 127.8.16.126:33311
I20260807 05:13:17.203413 28923 heartbeater.cc:464] Registering TS with master...
I20260807 05:13:17.203955 28923 heartbeater.cc:511] Master 127.8.16.126:33311 requested a full tablet report, sending...
I20260807 05:13:17.205046 28582 ts_manager.cc:194] Registered new tserver with Master: 755fd643af2d451e9d4bd64b9ad38da8 (127.8.16.66:37739)
I20260807 05:13:17.205663 28582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.66@KRBTEST.COM'} at 127.8.16.66:44777
WARNING: no policy specified for kudu/127.8.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
Entry for principal kudu/127.8.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab.
I20260807 05:13:17.222633 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:33311
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:17.330518 28928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:17.330781 28928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:17.330821 28928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:17.334817 28928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:17.334877 28928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:17.334942 28928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:13:17.340206 28928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.67.keytab
--principal=kudu/127.8.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:33311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:17.341260 28928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:17.342298 28928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:17.351356 28934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:17.351349 28936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:17.351424 28933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:17.351657 28928 server_base.cc:1034] running on GCE node
I20260807 05:13:17.352109 28928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:17.352732 28928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:17.353895 28928 hybrid_clock.cc:648] HybridClock initialized: now 1786079597353872 us; error 37 us; skew 500 ppm
Aug 07 05:13:17 dist-test-slave-49h3 krb5kdc[28532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079597, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:17.356612 28928 init.cc:377] Logged in from keytab as kudu/127.8.16.67@KRBTEST.COM (short username kudu)
I20260807 05:13:17.357553 28928 webserver.cc:460] Webserver started at http://127.8.16.67:36233/ using document root <none> and password file <none>
I20260807 05:13:17.358096 28928 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:17.358165 28928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:17.358378 28928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:17.360395 28928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "851024a00b384746946a1747e82fc14c"
format_stamp: "Formatted at 2026-08-07 05:13:17 on dist-test-slave-49h3"
server_key: "d802cc065a6ef48da4679db65ab0784e"
server_key_iv: "97fbe17c1725d3a45a1ba1dd048eab4d"
server_key_version: "encryptionkey@0"
I20260807 05:13:17.360934 28928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "851024a00b384746946a1747e82fc14c"
format_stamp: "Formatted at 2026-08-07 05:13:17 on dist-test-slave-49h3"
server_key: "d802cc065a6ef48da4679db65ab0784e"
server_key_iv: "97fbe17c1725d3a45a1ba1dd048eab4d"
server_key_version: "encryptionkey@0"
I20260807 05:13:17.364718 28928 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260807 05:13:17.367120 28945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:17.368335 28928 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:13:17.368448 28928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "851024a00b384746946a1747e82fc14c"
format_stamp: "Formatted at 2026-08-07 05:13:17 on dist-test-slave-49h3"
server_key: "d802cc065a6ef48da4679db65ab0784e"
server_key_iv: "97fbe17c1725d3a45a1ba1dd048eab4d"
server_key_version: "encryptionkey@0"
I20260807 05:13:17.368547 28928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:17.379150 28928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:17.381910 28928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:17.382088 28928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:17.382651 28928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:13:17.383440 28928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:13:17.383486 28928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:17.383538 28928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:13:17.383566 28928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:17.392117 28928 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:33521
I20260807 05:13:17.392132 29069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:33521 every 8 connection(s)
I20260807 05:13:17.393095 28928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:13:17.398840 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 28928
I20260807 05:13:17.398949 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:13:17.399210 8257 external_mini_cluster.cc:1428] Setting key f228e62c7044dea78e4db79c709a5264
Aug 07 05:13:17 dist-test-slave-49h3 krb5kdc[28532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079597, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.67@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:13:17.406349 28622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:17.394883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:35159 (local address 127.8.16.126:33311)
0807 05:13:17.395038 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:17.395042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:17.395834 (+ 792us) server_negotiation.cc:407] Connection header received
0807 05:13:17.397111 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:17.397114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:17.397175 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:17.397261 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:17.398477 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.399053 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.400317 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.400598 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.400686 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:17.403080 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:17.403083 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:17.403085 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:17.403109 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:17.404361 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:17.404829 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:17.404832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:17.404834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:17.404911 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:17.405222 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:17.405225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:17.405227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:17.405409 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:17.405538 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:17.406022 (+ 484us) server_negotiation.cc:299] Negotiation successful
0807 05:13:17.406177 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 05:13:17.407141 29073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:17.395238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:33311 (local address 127.8.16.67:35159)
0807 05:13:17.395676 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:17.395727 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:17.396902 (+ 1175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:17.397459 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:17.397468 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:17.397853 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:17.398333 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:17.398346 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.399166 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.399170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:17.400211 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:17.400221 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.400746 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.400750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:17.400904 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:17.401535 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:17.401551 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:17.402929 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:17.404464 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:17.404470 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:17.404485 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:17.404739 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:17.404991 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:17.404994 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:17.404996 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:17.405132 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:17.405535 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:17.405542 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:17.405797 (+ 255us) client_negotiation.cc:769] Sending connection context
0807 05:13:17.405964 (+ 167us) client_negotiation.cc:240] Negotiation successful
0807 05:13:17.406311 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":118,"threads_started":1}
I20260807 05:13:17.408174 29071 heartbeater.cc:347] Connected to a master server at 127.8.16.126:33311
I20260807 05:13:17.408421 29071 heartbeater.cc:464] Registering TS with master...
I20260807 05:13:17.408910 29071 heartbeater.cc:511] Master 127.8.16.126:33311 requested a full tablet report, sending...
I20260807 05:13:17.409927 28582 ts_manager.cc:194] Registered new tserver with Master: 851024a00b384746946a1747e82fc14c (127.8.16.67:33521)
I20260807 05:13:17.410481 28582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.16.67@KRBTEST.COM'} at 127.8.16.67:35159
I20260807 05:13:17.412863 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:13:17.421730 28622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:17.414806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56324 (local address 127.8.16.126:33311)
0807 05:13:17.414924 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:17.414929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:17.415081 (+ 152us) server_negotiation.cc:407] Connection header received
0807 05:13:17.415201 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:17.415214 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:17.415256 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:17.415316 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:17.415975 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.416479 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.417281 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.417545 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.417631 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:17.418922 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:17.418926 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:17.418929 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:17.418955 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:17.420162 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:17.420534 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:17.420537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:17.420539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:17.420617 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:17.420860 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:17.420863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:17.420865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:17.421082 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:17.421198 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:17.421467 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 05:13:17.421579 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 05:13:17.424445 28582 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:13:17.427402 28582 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:13:17.441394 29083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:17.435439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:37739 (local address 127.0.0.1:60986)
0807 05:13:17.436011 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:17.436049 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:17.436172 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:17.437438 (+ 1266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:17.437440 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:17.437473 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:17.437542 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:17.437547 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.439071 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.439073 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:17.440051 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:17.440061 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.440986 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.440990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:17.441083 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:17.441100 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 05:13:17.441176 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 05:13:17.441266 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":16,"thread_start_us":65,"threads_started":1}
I20260807 05:13:17.441521 29084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:17.435582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60986 (local address 127.8.16.66:37739)
0807 05:13:17.436839 (+ 1257us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:17.436850 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:17.436871 (+ 21us) server_negotiation.cc:407] Connection header received
0807 05:13:17.436943 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:17.436948 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:17.437133 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:17.437309 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:17.437677 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.438894 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.440170 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.440849 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.441036 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:17.441189 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:17.441240 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 05:13:17.441331 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1158,"thread_start_us":147,"threads_started":1}
I20260807 05:13:17.441843 29082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:17.435499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:46365 (local address 127.0.0.1:42764)
0807 05:13:17.436394 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:17.436407 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:17.436481 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:17.437310 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:17.437316 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:17.437338 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:17.437526 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:17.437535 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.439031 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.439035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:17.440377 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:17.440387 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.441417 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.441420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:17.441532 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:17.441555 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:13:17.441649 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 05:13:17.441717 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":837,"mutex_wait_us":246,"thread_start_us":123,"threads_started":1}
I20260807 05:13:17.442226 29086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:17.436130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42764 (local address 127.8.16.65:46365)
0807 05:13:17.436839 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:17.436850 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:17.436871 (+ 21us) server_negotiation.cc:407] Connection header received
0807 05:13:17.436943 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:17.436948 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:17.437126 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:17.437308 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:17.437677 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.438891 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.440502 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.441298 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.441403 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:17.441559 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:17.442004 (+ 445us) server_negotiation.cc:299] Negotiation successful
0807 05:13:17.442100 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":149,"threads_started":1}
I20260807 05:13:17.442651 29085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:17.436137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:33521 (local address 127.0.0.1:50228)
0807 05:13:17.436483 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:17.436496 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:17.436602 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:17.437333 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:17.437336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:17.437349 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:17.437676 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:17.437681 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.438633 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.438636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:17.439838 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:17.439849 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.440848 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.440852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:17.440965 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:17.442372 (+ 1407us) client_negotiation.cc:769] Sending connection context
0807 05:13:17.442448 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 05:13:17.442522 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":225,"spinlock_wait_cycles":2944,"thread_start_us":113,"threads_started":1}
I20260807 05:13:17.442807 29087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:17.436324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50228 (local address 127.8.16.67:33521)
0807 05:13:17.436696 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:17.436708 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:17.436732 (+ 24us) server_negotiation.cc:407] Connection header received
0807 05:13:17.436804 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:17.436809 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:17.437013 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:17.437157 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:17.437771 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.438511 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.439977 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.440741 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.440834 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:17.440981 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:17.442485 (+ 1504us) server_negotiation.cc:299] Negotiation successful
0807 05:13:17.442580 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":132,"threads_started":1}
I20260807 05:13:17.444782 28853 tablet_service.cc:1467] Processing CreateTablet for tablet ebc6c8279d9f483d85dd7bc2758284c7 (DEFAULT_TABLE table=test-table [id=1c8515b98b2e49cab0322466e5bc5b3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:13:17.444787 28713 tablet_service.cc:1467] Processing CreateTablet for tablet ebc6c8279d9f483d85dd7bc2758284c7 (DEFAULT_TABLE table=test-table [id=1c8515b98b2e49cab0322466e5bc5b3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:13:17.445336 28994 tablet_service.cc:1467] Processing CreateTablet for tablet ebc6c8279d9f483d85dd7bc2758284c7 (DEFAULT_TABLE table=test-table [id=1c8515b98b2e49cab0322466e5bc5b3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:13:17.445946 28853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebc6c8279d9f483d85dd7bc2758284c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:17.445946 28713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebc6c8279d9f483d85dd7bc2758284c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:17.446238 28994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebc6c8279d9f483d85dd7bc2758284c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:17.452602 29088 tablet_bootstrap.cc:492] T ebc6c8279d9f483d85dd7bc2758284c7 P 851024a00b384746946a1747e82fc14c: Bootstrap starting.
I20260807 05:13:17.453398 29089 tablet_bootstrap.cc:492] T ebc6c8279d9f483d85dd7bc2758284c7 P 755fd643af2d451e9d4bd64b9ad38da8: Bootstrap starting.
I20260807 05:13:17.453776 29090 tablet_bootstrap.cc:492] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2: Bootstrap starting.
I20260807 05:13:17.454598 29088 tablet_bootstrap.cc:654] T ebc6c8279d9f483d85dd7bc2758284c7 P 851024a00b384746946a1747e82fc14c: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:17.455578 29088 log.cc:824] T ebc6c8279d9f483d85dd7bc2758284c7 P 851024a00b384746946a1747e82fc14c: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:17.455914 29089 tablet_bootstrap.cc:654] T ebc6c8279d9f483d85dd7bc2758284c7 P 755fd643af2d451e9d4bd64b9ad38da8: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:17.456100 29090 tablet_bootstrap.cc:654] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:17.456746 29089 log.cc:824] T ebc6c8279d9f483d85dd7bc2758284c7 P 755fd643af2d451e9d4bd64b9ad38da8: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:17.456770 29090 log.cc:824] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:17.457379 29088 tablet_bootstrap.cc:492] T ebc6c8279d9f483d85dd7bc2758284c7 P 851024a00b384746946a1747e82fc14c: No bootstrap required, opened a new log
I20260807 05:13:17.457580 29088 ts_tablet_manager.cc:1397] T ebc6c8279d9f483d85dd7bc2758284c7 P 851024a00b384746946a1747e82fc14c: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260807 05:13:17.458631 29090 tablet_bootstrap.cc:492] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2: No bootstrap required, opened a new log
I20260807 05:13:17.458631 29089 tablet_bootstrap.cc:492] T ebc6c8279d9f483d85dd7bc2758284c7 P 755fd643af2d451e9d4bd64b9ad38da8: No bootstrap required, opened a new log
I20260807 05:13:17.458812 29089 ts_tablet_manager.cc:1397] T ebc6c8279d9f483d85dd7bc2758284c7 P 755fd643af2d451e9d4bd64b9ad38da8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 05:13:17.458812 29090 ts_tablet_manager.cc:1397] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260807 05:13:17.460444 29088 raft_consensus.cc:348] T ebc6c8279d9f483d85dd7bc2758284c7 P 851024a00b384746946a1747e82fc14c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755fd643af2d451e9d4bd64b9ad38da8" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37739 } } peers { permanent_uuid: "d348fb0a5ffe4c7bb3fbce39609960f2" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 46365 } } peers { permanent_uuid: "851024a00b384746946a1747e82fc14c" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 33521 } }
I20260807 05:13:17.460662 29088 raft_consensus.cc:374] T ebc6c8279d9f483d85dd7bc2758284c7 P 851024a00b384746946a1747e82fc14c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:17.460743 29088 raft_consensus.cc:729] T ebc6c8279d9f483d85dd7bc2758284c7 P 851024a00b384746946a1747e82fc14c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 851024a00b384746946a1747e82fc14c, State: Initialized, Role: FOLLOWER
I20260807 05:13:17.461148 29088 consensus_queue.cc:260] T ebc6c8279d9f483d85dd7bc2758284c7 P 851024a00b384746946a1747e82fc14c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755fd643af2d451e9d4bd64b9ad38da8" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37739 } } peers { permanent_uuid: "d348fb0a5ffe4c7bb3fbce39609960f2" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 46365 } } peers { permanent_uuid: "851024a00b384746946a1747e82fc14c" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 33521 } }
I20260807 05:13:17.461822 29071 heartbeater.cc:503] Master 127.8.16.126:33311 was elected leader, sending a full tablet report...
I20260807 05:13:17.462054 29088 ts_tablet_manager.cc:1428] T ebc6c8279d9f483d85dd7bc2758284c7 P 851024a00b384746946a1747e82fc14c: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 05:13:17.462484 29090 raft_consensus.cc:348] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755fd643af2d451e9d4bd64b9ad38da8" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37739 } } peers { permanent_uuid: "d348fb0a5ffe4c7bb3fbce39609960f2" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 46365 } } peers { permanent_uuid: "851024a00b384746946a1747e82fc14c" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 33521 } }
I20260807 05:13:17.462746 29090 raft_consensus.cc:374] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:17.462844 29090 raft_consensus.cc:729] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d348fb0a5ffe4c7bb3fbce39609960f2, State: Initialized, Role: FOLLOWER
I20260807 05:13:17.463330 29090 consensus_queue.cc:260] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755fd643af2d451e9d4bd64b9ad38da8" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37739 } } peers { permanent_uuid: "d348fb0a5ffe4c7bb3fbce39609960f2" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 46365 } } peers { permanent_uuid: "851024a00b384746946a1747e82fc14c" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 33521 } }
I20260807 05:13:17.463601 29089 raft_consensus.cc:348] T ebc6c8279d9f483d85dd7bc2758284c7 P 755fd643af2d451e9d4bd64b9ad38da8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755fd643af2d451e9d4bd64b9ad38da8" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37739 } } peers { permanent_uuid: "d348fb0a5ffe4c7bb3fbce39609960f2" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 46365 } } peers { permanent_uuid: "851024a00b384746946a1747e82fc14c" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 33521 } }
I20260807 05:13:17.463901 29089 raft_consensus.cc:374] T ebc6c8279d9f483d85dd7bc2758284c7 P 755fd643af2d451e9d4bd64b9ad38da8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:17.463989 29089 raft_consensus.cc:729] T ebc6c8279d9f483d85dd7bc2758284c7 P 755fd643af2d451e9d4bd64b9ad38da8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 755fd643af2d451e9d4bd64b9ad38da8, State: Initialized, Role: FOLLOWER
I20260807 05:13:17.464304 29090 ts_tablet_manager.cc:1428] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 05:13:17.464656 29089 consensus_queue.cc:260] T ebc6c8279d9f483d85dd7bc2758284c7 P 755fd643af2d451e9d4bd64b9ad38da8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755fd643af2d451e9d4bd64b9ad38da8" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37739 } } peers { permanent_uuid: "d348fb0a5ffe4c7bb3fbce39609960f2" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 46365 } } peers { permanent_uuid: "851024a00b384746946a1747e82fc14c" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 33521 } }
I20260807 05:13:17.464776 28781 heartbeater.cc:503] Master 127.8.16.126:33311 was elected leader, sending a full tablet report...
I20260807 05:13:17.465322 28923 heartbeater.cc:503] Master 127.8.16.126:33311 was elected leader, sending a full tablet report...
I20260807 05:13:17.465646 29089 ts_tablet_manager.cc:1428] T ebc6c8279d9f483d85dd7bc2758284c7 P 755fd643af2d451e9d4bd64b9ad38da8: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
W20260807 05:13:17.491919 28782 tablet.cc:2369] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 05:13:17.644151 29072 tablet.cc:2369] T ebc6c8279d9f483d85dd7bc2758284c7 P 851024a00b384746946a1747e82fc14c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:13:17.677232 29095 raft_consensus.cc:482] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:13:17.677438 29095 raft_consensus.cc:504] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755fd643af2d451e9d4bd64b9ad38da8" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37739 } } peers { permanent_uuid: "d348fb0a5ffe4c7bb3fbce39609960f2" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 46365 } } peers { permanent_uuid: "851024a00b384746946a1747e82fc14c" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 33521 } }
I20260807 05:13:17.678557 29095 leader_election.cc:288] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 755fd643af2d451e9d4bd64b9ad38da8 (127.8.16.66:37739), 851024a00b384746946a1747e82fc14c (127.8.16.67:33521)
I20260807 05:13:17.683595 29087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:17.679312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:57397 (local address 127.8.16.67:33521)
0807 05:13:17.679433 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:17.679436 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:17.679488 (+ 52us) server_negotiation.cc:407] Connection header received
0807 05:13:17.679619 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:17.679622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:17.679671 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:17.679739 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:17.679972 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.680750 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.682358 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.682966 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.683080 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:17.683135 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:17.683388 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 05:13:17.683467 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 05:13:17.683596 29084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:17.678941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:45131 (local address 127.8.16.66:37739)
0807 05:13:17.679084 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:17.679088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:17.679182 (+ 94us) server_negotiation.cc:407] Connection header received
0807 05:13:17.679378 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:17.679382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:17.679429 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:17.679478 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:17.679875 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.680656 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.682358 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.682966 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.683070 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:17.683135 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:17.683388 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 05:13:17.683468 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 05:13:17.683606 29098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:17.679168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:33521 (local address 127.8.16.65:57397)
0807 05:13:17.679405 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:17.679416 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:17.679517 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:17.679756 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:17.679758 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:17.679769 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:17.679879 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:17.679883 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.680843 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.680846 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:17.682225 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:17.682236 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.683078 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.683082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:17.683208 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:17.683259 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 05:13:17.683326 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 05:13:17.683420 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":110,"thread_start_us":79,"threads_started":1}
I20260807 05:13:17.683609 29097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:17.678795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:37739 (local address 127.8.16.65:45131)
0807 05:13:17.679088 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:17.679103 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:17.679240 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:17.679634 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:17.679637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:17.679653 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:17.679778 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:17.679785 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.680749 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.680751 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:17.682225 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:17.682236 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.683078 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.683082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:17.683201 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:17.683250 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 05:13:17.683326 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 05:13:17.683420 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":101,"thread_start_us":95,"threads_started":1}
I20260807 05:13:17.684219 28875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebc6c8279d9f483d85dd7bc2758284c7" candidate_uuid: "d348fb0a5ffe4c7bb3fbce39609960f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "755fd643af2d451e9d4bd64b9ad38da8" is_pre_election: true
I20260807 05:13:17.684545 28875 raft_consensus.cc:2454] T ebc6c8279d9f483d85dd7bc2758284c7 P 755fd643af2d451e9d4bd64b9ad38da8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d348fb0a5ffe4c7bb3fbce39609960f2 in term 0.
I20260807 05:13:17.684571 29015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebc6c8279d9f483d85dd7bc2758284c7" candidate_uuid: "d348fb0a5ffe4c7bb3fbce39609960f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "851024a00b384746946a1747e82fc14c" is_pre_election: true
I20260807 05:13:17.684832 29015 raft_consensus.cc:2454] T ebc6c8279d9f483d85dd7bc2758284c7 P 851024a00b384746946a1747e82fc14c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d348fb0a5ffe4c7bb3fbce39609960f2 in term 0.
I20260807 05:13:17.685072 28665 leader_election.cc:302] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 755fd643af2d451e9d4bd64b9ad38da8, d348fb0a5ffe4c7bb3fbce39609960f2; no voters:
I20260807 05:13:17.685321 29095 raft_consensus.cc:2781] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:13:17.685407 29095 raft_consensus.cc:482] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:13:17.685488 29095 raft_consensus.cc:3037] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:17.686522 29095 raft_consensus.cc:504] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755fd643af2d451e9d4bd64b9ad38da8" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37739 } } peers { permanent_uuid: "d348fb0a5ffe4c7bb3fbce39609960f2" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 46365 } } peers { permanent_uuid: "851024a00b384746946a1747e82fc14c" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 33521 } }
I20260807 05:13:17.686836 29095 leader_election.cc:288] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2 [CANDIDATE]: Term 1 election: Requested vote from peers 755fd643af2d451e9d4bd64b9ad38da8 (127.8.16.66:37739), 851024a00b384746946a1747e82fc14c (127.8.16.67:33521)
I20260807 05:13:17.687207 28875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebc6c8279d9f483d85dd7bc2758284c7" candidate_uuid: "d348fb0a5ffe4c7bb3fbce39609960f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "755fd643af2d451e9d4bd64b9ad38da8"
I20260807 05:13:17.687253 29015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebc6c8279d9f483d85dd7bc2758284c7" candidate_uuid: "d348fb0a5ffe4c7bb3fbce39609960f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "851024a00b384746946a1747e82fc14c"
I20260807 05:13:17.687353 29015 raft_consensus.cc:3037] T ebc6c8279d9f483d85dd7bc2758284c7 P 851024a00b384746946a1747e82fc14c [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:17.687340 28875 raft_consensus.cc:3037] T ebc6c8279d9f483d85dd7bc2758284c7 P 755fd643af2d451e9d4bd64b9ad38da8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:17.688557 29015 raft_consensus.cc:2454] T ebc6c8279d9f483d85dd7bc2758284c7 P 851024a00b384746946a1747e82fc14c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d348fb0a5ffe4c7bb3fbce39609960f2 in term 1.
I20260807 05:13:17.688557 28875 raft_consensus.cc:2454] T ebc6c8279d9f483d85dd7bc2758284c7 P 755fd643af2d451e9d4bd64b9ad38da8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d348fb0a5ffe4c7bb3fbce39609960f2 in term 1.
I20260807 05:13:17.688854 28665 leader_election.cc:302] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 851024a00b384746946a1747e82fc14c, d348fb0a5ffe4c7bb3fbce39609960f2; no voters:
I20260807 05:13:17.689008 29095 raft_consensus.cc:2781] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:17.689235 29095 raft_consensus.cc:686] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2 [term 1 LEADER]: Becoming Leader. State: Replica: d348fb0a5ffe4c7bb3fbce39609960f2, State: Running, Role: LEADER
W20260807 05:13:17.689567 28924 tablet.cc:2369] T ebc6c8279d9f483d85dd7bc2758284c7 P 755fd643af2d451e9d4bd64b9ad38da8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:13:17.689592 29095 consensus_queue.cc:237] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755fd643af2d451e9d4bd64b9ad38da8" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37739 } } peers { permanent_uuid: "d348fb0a5ffe4c7bb3fbce39609960f2" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 46365 } } peers { permanent_uuid: "851024a00b384746946a1747e82fc14c" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 33521 } }
I20260807 05:13:17.692574 28581 catalog_manager.cc:5495] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to d348fb0a5ffe4c7bb3fbce39609960f2 (127.8.16.65). New cstate: current_term: 1 leader_uuid: "d348fb0a5ffe4c7bb3fbce39609960f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755fd643af2d451e9d4bd64b9ad38da8" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d348fb0a5ffe4c7bb3fbce39609960f2" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 46365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "851024a00b384746946a1747e82fc14c" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 33521 } health_report { overall_health: UNKNOWN } } }
I20260807 05:13:17.732110 29086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:17.728940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42772 (local address 127.8.16.65:46365)
0807 05:13:17.729076 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:17.729080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:17.729093 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:13:17.729168 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:17.729170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:17.729225 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:17.729306 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:13:17.729551 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.730058 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.730848 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.731058 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:17.731131 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:17.731350 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:13:17.731776 (+ 426us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:13:17.731862 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:17.731924 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 05:13:17.731972 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 05:13:17.737910 28875 raft_consensus.cc:1261] T ebc6c8279d9f483d85dd7bc2758284c7 P 755fd643af2d451e9d4bd64b9ad38da8 [term 1 FOLLOWER]: Refusing update from remote peer d348fb0a5ffe4c7bb3fbce39609960f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:13:17.737910 29015 raft_consensus.cc:1261] T ebc6c8279d9f483d85dd7bc2758284c7 P 851024a00b384746946a1747e82fc14c [term 1 FOLLOWER]: Refusing update from remote peer d348fb0a5ffe4c7bb3fbce39609960f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:13:17.738615 29099 consensus_queue.cc:1035] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "755fd643af2d451e9d4bd64b9ad38da8" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 37739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:13:17.738792 29095 consensus_queue.cc:1035] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "851024a00b384746946a1747e82fc14c" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 33521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:13:17.745798 29102 mvcc.cc:205] Tried to move back new op lower bound from 7315782032327217152 to 7315782032139382784. Current Snapshot: MvccSnapshot[applied={T|T < 7315782032327217152}]
I20260807 05:13:17.747777 29103 mvcc.cc:205] Tried to move back new op lower bound from 7315782032327217152 to 7315782032139382784. Current Snapshot: MvccSnapshot[applied={T|T < 7315782032327217152}]
I20260807 05:13:17.747951 29105 mvcc.cc:205] Tried to move back new op lower bound from 7315782032327217152 to 7315782032139382784. Current Snapshot: MvccSnapshot[applied={T|T < 7315782032327217152}]
I20260807 05:13:17.754288 28581 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:13:17.754482 28581 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:13:17.756969 28581 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dcfe521b3f084ce7a0c5f975979dabd8: Sending DeleteTablet for 3 replicas of tablet ebc6c8279d9f483d85dd7bc2758284c7
I20260807 05:13:17.757732 28713 tablet_service.cc:1514] Processing DeleteTablet for tablet ebc6c8279d9f483d85dd7bc2758284c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:13:17 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:42764
I20260807 05:13:17.757795 28853 tablet_service.cc:1514] Processing DeleteTablet for tablet ebc6c8279d9f483d85dd7bc2758284c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:13:17 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:60986
I20260807 05:13:17.757946 28994 tablet_service.cc:1514] Processing DeleteTablet for tablet ebc6c8279d9f483d85dd7bc2758284c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:13:17 UTC) from {username='kudu', principal='kudu/127.8.16.126@KRBTEST.COM'} at 127.0.0.1:50228
I20260807 05:13:17.758401 29115 tablet_replica.cc:330] T ebc6c8279d9f483d85dd7bc2758284c7 P 851024a00b384746946a1747e82fc14c: stopping tablet replica
I20260807 05:13:17.758509 29113 tablet_replica.cc:330] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2: stopping tablet replica
I20260807 05:13:17.758793 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 28645
I20260807 05:13:17.758796 29115 raft_consensus.cc:2229] T ebc6c8279d9f483d85dd7bc2758284c7 P 851024a00b384746946a1747e82fc14c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:13:17.758796 29113 raft_consensus.cc:2229] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:13:17.759088 29115 raft_consensus.cc:2258] T ebc6c8279d9f483d85dd7bc2758284c7 P 851024a00b384746946a1747e82fc14c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:13:17.759145 29113 raft_consensus.cc:2258] T ebc6c8279d9f483d85dd7bc2758284c7 P d348fb0a5ffe4c7bb3fbce39609960f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:13:17.759397 29114 tablet_replica.cc:330] T ebc6c8279d9f483d85dd7bc2758284c7 P 755fd643af2d451e9d4bd64b9ad38da8: stopping tablet replica
I20260807 05:13:17.759665 29114 raft_consensus.cc:2229] T ebc6c8279d9f483d85dd7bc2758284c7 P 755fd643af2d451e9d4bd64b9ad38da8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:13:17.759877 29114 raft_consensus.cc:2258] T ebc6c8279d9f483d85dd7bc2758284c7 P 755fd643af2d451e9d4bd64b9ad38da8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:13:17.761667 29115 ts_tablet_manager.cc:1905] T ebc6c8279d9f483d85dd7bc2758284c7 P 851024a00b384746946a1747e82fc14c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:13:17.762449 29114 ts_tablet_manager.cc:1905] T ebc6c8279d9f483d85dd7bc2758284c7 P 755fd643af2d451e9d4bd64b9ad38da8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:13:17.764328 29115 ts_tablet_manager.cc:1918] T ebc6c8279d9f483d85dd7bc2758284c7 P 851024a00b384746946a1747e82fc14c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:13:17.764420 29115 log.cc:1196] T ebc6c8279d9f483d85dd7bc2758284c7 P 851024a00b384746946a1747e82fc14c: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/ebc6c8279d9f483d85dd7bc2758284c7
I20260807 05:13:17.764783 29115 ts_tablet_manager.cc:1939] T ebc6c8279d9f483d85dd7bc2758284c7 P 851024a00b384746946a1747e82fc14c: Deleting consensus metadata
I20260807 05:13:17.764935 29114 ts_tablet_manager.cc:1918] T ebc6c8279d9f483d85dd7bc2758284c7 P 755fd643af2d451e9d4bd64b9ad38da8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:13:17.765022 29114 log.cc:1196] T ebc6c8279d9f483d85dd7bc2758284c7 P 755fd643af2d451e9d4bd64b9ad38da8: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079354071073-8257-0/minicluster-data/ts-1/wal/wals/ebc6c8279d9f483d85dd7bc2758284c7
I20260807 05:13:17.765295 29114 ts_tablet_manager.cc:1939] T ebc6c8279d9f483d85dd7bc2758284c7 P 755fd643af2d451e9d4bd64b9ad38da8: Deleting consensus metadata
I20260807 05:13:17.766078 28566 catalog_manager.cc:4841] TS 851024a00b384746946a1747e82fc14c (127.8.16.67:33521): tablet ebc6c8279d9f483d85dd7bc2758284c7 (table test-table [id=1c8515b98b2e49cab0322466e5bc5b3f]) successfully deleted
W20260807 05:13:17.766247 28563 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.16.65:46365 (error 108)
I20260807 05:13:17.766484 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 28787
I20260807 05:13:17.766532 28566 catalog_manager.cc:4841] TS 755fd643af2d451e9d4bd64b9ad38da8 (127.8.16.66:37739): tablet ebc6c8279d9f483d85dd7bc2758284c7 (table test-table [id=1c8515b98b2e49cab0322466e5bc5b3f]) successfully deleted
I20260807 05:13:17.768329 29085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:17.768002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:46365 (local address 127.0.0.1:42780)
0807 05:13:17.768193 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:17.768259 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.65:46365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
W20260807 05:13:17.768456 28563 catalog_manager.cc:4584] TS d348fb0a5ffe4c7bb3fbce39609960f2 (127.8.16.65:46365): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ebc6c8279d9f483d85dd7bc2758284c7: Network error: Client connection negotiation failed: client connection to 127.8.16.65:46365: connect: Connection refused (error 111)
I20260807 05:13:17.773070 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 28928
I20260807 05:13:17.778183 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 28547
2026-08-07T05:13:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2327 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T05:13:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:17Z Disabled control of system clock
I20260807 05:13:17.807986 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:36143
--webserver_interface=127.8.16.126
--webserver_port=0
--builtin_ntp_servers=127.8.16.84:36095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:36143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 05:13:17.914642 29120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:17.914918 29120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:17.914955 29120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:17.918895 29120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:17.918956 29120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:17.918970 29120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:17.918985 29120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:17.919001 29120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:17.923749 29120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:36095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:36143
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:36143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:17.924803 29120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:17.926164 29120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:17.934311 29126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:17.934316 29129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:17.934316 29127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:17.934644 29120 server_base.cc:1034] running on GCE node
I20260807 05:13:17.935281 29120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:17.936673 29120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:17.937847 29120 hybrid_clock.cc:648] HybridClock initialized: now 1786079597937824 us; error 36 us; skew 500 ppm
I20260807 05:13:17.940250 29120 webserver.cc:460] Webserver started at http://127.8.16.126:44869/ using document root <none> and password file <none>
I20260807 05:13:17.940934 29120 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:17.941004 29120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:17.941221 29120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:17.944161 29120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "5feb561d53eb4a77a3630ce831a543e3"
format_stamp: "Formatted at 2026-08-07 05:13:17 on dist-test-slave-49h3"
server_key: "5d017829ff75cc4f80cc5c002f18f3bb"
server_key_iv: "d4981349706021e148bbafae54059f4c"
server_key_version: "encryptionkey@0"
I20260807 05:13:17.944718 29120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "5feb561d53eb4a77a3630ce831a543e3"
format_stamp: "Formatted at 2026-08-07 05:13:17 on dist-test-slave-49h3"
server_key: "5d017829ff75cc4f80cc5c002f18f3bb"
server_key_iv: "d4981349706021e148bbafae54059f4c"
server_key_version: "encryptionkey@0"
I20260807 05:13:17.948608 29120 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 05:13:17.951025 29136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:17.952171 29120 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:13:17.952291 29120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "5feb561d53eb4a77a3630ce831a543e3"
format_stamp: "Formatted at 2026-08-07 05:13:17 on dist-test-slave-49h3"
server_key: "5d017829ff75cc4f80cc5c002f18f3bb"
server_key_iv: "d4981349706021e148bbafae54059f4c"
server_key_version: "encryptionkey@0"
I20260807 05:13:17.952384 29120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:17.975979 29120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:17.976716 29120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:17.976878 29120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:17.983726 29120 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:36143
I20260807 05:13:17.983736 29191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:36143 every 8 connection(s)
I20260807 05:13:17.984683 29120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:17.987885 29192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:17.993489 29192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3: Bootstrap starting.
I20260807 05:13:17.993878 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 29120
I20260807 05:13:17.994006 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:17.994261 8257 external_mini_cluster.cc:1428] Setting key 772b5203d55fe665aae6762a0532d991
I20260807 05:13:17.996047 29192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:17.996757 29192 log.cc:824] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:17.998842 29192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3: No bootstrap required, opened a new log
I20260807 05:13:18.002843 29192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5feb561d53eb4a77a3630ce831a543e3" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36143 } }
I20260807 05:13:18.003065 29192 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:18.003129 29192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5feb561d53eb4a77a3630ce831a543e3, State: Initialized, Role: FOLLOWER
I20260807 05:13:18.003338 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:17.995612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60934 (local address 127.8.16.126:36143)
0807 05:13:17.996244 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:17.996259 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:17.996294 (+ 35us) server_negotiation.cc:407] Connection header received
0807 05:13:17.996916 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:17.996938 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:17.997401 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:17.997744 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:17.998069 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:17.999037 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.000229 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.000696 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.000806 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:18.001649 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:18.001654 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:18.001666 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:18.002031 (+ 365us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:18.002086 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:18.002105 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:18.002177 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:18.002310 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 05:13:18.002571 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":120,"threads_started":1}
I20260807 05:13:18.003582 29192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5feb561d53eb4a77a3630ce831a543e3" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36143 } }
I20260807 05:13:18.003724 29192 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:18.003790 29192 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:18.003883 29192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:18.004833 29192 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5feb561d53eb4a77a3630ce831a543e3" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36143 } }
I20260807 05:13:18.005156 29192 leader_election.cc:302] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5feb561d53eb4a77a3630ce831a543e3; no voters:
I20260807 05:13:18.005493 29192 leader_election.cc:288] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:18.005559 29197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:18.005762 29197 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3 [term 1 LEADER]: Becoming Leader. State: Replica: 5feb561d53eb4a77a3630ce831a543e3, State: Running, Role: LEADER
I20260807 05:13:18.006083 29197 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5feb561d53eb4a77a3630ce831a543e3" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36143 } }
I20260807 05:13:18.006595 29192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:18.007575 29198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5feb561d53eb4a77a3630ce831a543e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5feb561d53eb4a77a3630ce831a543e3" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36143 } } }
I20260807 05:13:18.007575 29199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5feb561d53eb4a77a3630ce831a543e3. Latest consensus state: current_term: 1 leader_uuid: "5feb561d53eb4a77a3630ce831a543e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5feb561d53eb4a77a3630ce831a543e3" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36143 } } }
I20260807 05:13:18.007716 29198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:18.007719 29199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:18.008610 29206 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:18.011569 29206 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:18.017357 29206 catalog_manager.cc:1347] Generated new cluster ID: f98928949a784902ab33713e11f980f9
I20260807 05:13:18.017431 29206 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:18.030282 29206 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:18.031486 29206 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:18.037134 29206 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3: Generated new TSK 0
I20260807 05:13:18.037827 29206 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:13:18.044797 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.65:0
--local_ip_for_outbound_sockets=127.8.16.65
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36143
--builtin_ntp_servers=127.8.16.84:36095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 05:13:18.154820 29217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:18.155088 29217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:18.155141 29217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:18.159257 29217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:18.159322 29217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:18.159397 29217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.65
I20260807 05:13:18.164124 29217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:36095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.16.65
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.65
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:18.165180 29217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:18.166586 29217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:18.175773 29222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:18.175752 29223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:18.175745 29225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:18.176020 29217 server_base.cc:1034] running on GCE node
I20260807 05:13:18.176433 29217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:18.176996 29217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:18.178170 29217 hybrid_clock.cc:648] HybridClock initialized: now 1786079598178148 us; error 37 us; skew 500 ppm
I20260807 05:13:18.179919 29217 webserver.cc:460] Webserver started at http://127.8.16.65:44043/ using document root <none> and password file <none>
I20260807 05:13:18.180490 29217 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:18.180557 29217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:18.180749 29217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:18.182910 29217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-0/data/instance:
uuid: "ccf491001b4d431da5bb20c3011b5c23"
format_stamp: "Formatted at 2026-08-07 05:13:18 on dist-test-slave-49h3"
server_key: "c8d6494c0fcdf810ce0b3302d8c940ec"
server_key_iv: "cf3526f55b4d9ad5e95d409003bd8cdd"
server_key_version: "encryptionkey@0"
I20260807 05:13:18.183456 29217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance:
uuid: "ccf491001b4d431da5bb20c3011b5c23"
format_stamp: "Formatted at 2026-08-07 05:13:18 on dist-test-slave-49h3"
server_key: "c8d6494c0fcdf810ce0b3302d8c940ec"
server_key_iv: "cf3526f55b4d9ad5e95d409003bd8cdd"
server_key_version: "encryptionkey@0"
I20260807 05:13:18.187126 29217 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260807 05:13:18.189422 29231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:18.190665 29217 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:13:18.190796 29217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-0/wal
uuid: "ccf491001b4d431da5bb20c3011b5c23"
format_stamp: "Formatted at 2026-08-07 05:13:18 on dist-test-slave-49h3"
server_key: "c8d6494c0fcdf810ce0b3302d8c940ec"
server_key_iv: "cf3526f55b4d9ad5e95d409003bd8cdd"
server_key_version: "encryptionkey@0"
I20260807 05:13:18.190903 29217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:18.205999 29217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:18.206811 29217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:18.206993 29217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:18.207617 29217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:13:18.208648 29217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:13:18.208706 29217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:18.208758 29217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:13:18.208787 29217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:18.217329 29217 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.65:43723
I20260807 05:13:18.217355 29351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.65:43723 every 8 connection(s)
I20260807 05:13:18.218235 29217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-0/data/info.pb
I20260807 05:13:18.221702 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 29217
I20260807 05:13:18.221812 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-0/wal/instance
I20260807 05:13:18.222077 8257 external_mini_cluster.cc:1428] Setting key e2fc636625e7d23ae4211928f2e36ac6
I20260807 05:13:18.224136 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.66:0
--local_ip_for_outbound_sockets=127.8.16.66
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36143
--builtin_ntp_servers=127.8.16.84:36095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 05:13:18.227541 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:18.219962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.65:52025 (local address 127.8.16.126:36143)
0807 05:13:18.220088 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:18.220092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:18.221124 (+ 1032us) server_negotiation.cc:407] Connection header received
0807 05:13:18.222585 (+ 1461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:18.222589 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:18.222654 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:18.222744 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:18.223811 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.224540 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.225716 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.225944 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.226034 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:18.227091 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:18.227095 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:18.227098 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:18.227112 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:18.227131 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:18.227139 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:18.227203 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:18.227339 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 05:13:18.227398 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 05:13:18.228387 29354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:18.220360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36143 (local address 127.8.16.65:52025)
0807 05:13:18.220978 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:18.221019 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:18.222351 (+ 1332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:18.222907 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:18.222916 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:18.223022 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:18.223668 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:18.223685 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.224672 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.224675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:18.225590 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:18.225600 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.226100 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.226105 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:18.226268 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:18.226848 (+ 580us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:13:18.226850 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:18.226889 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:13:18.226953 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:18.227214 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:18.227223 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:18.227229 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 05:13:18.227332 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 05:13:18.227607 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":117,"threads_started":1}
I20260807 05:13:18.229563 29352 heartbeater.cc:347] Connected to a master server at 127.8.16.126:36143
I20260807 05:13:18.229787 29352 heartbeater.cc:464] Registering TS with master...
I20260807 05:13:18.230283 29352 heartbeater.cc:511] Master 127.8.16.126:36143 requested a full tablet report, sending...
I20260807 05:13:18.231696 29154 ts_manager.cc:194] Registered new tserver with Master: ccf491001b4d431da5bb20c3011b5c23 (127.8.16.65:43723)
I20260807 05:13:18.232781 29154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.16.65:52025
W20260807 05:13:18.332419 29355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:18.332688 29355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:18.332744 29355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:18.336743 29355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:18.336805 29355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:18.336879 29355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.66
I20260807 05:13:18.341600 29355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:36095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.16.66
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.66
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:18.342675 29355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:18.344027 29355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:18.353497 29363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:18.353515 29361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:18.353497 29360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:18.353817 29355 server_base.cc:1034] running on GCE node
I20260807 05:13:18.354192 29355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:18.354758 29355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:18.355908 29355 hybrid_clock.cc:648] HybridClock initialized: now 1786079598355893 us; error 24 us; skew 500 ppm
I20260807 05:13:18.357679 29355 webserver.cc:460] Webserver started at http://127.8.16.66:35463/ using document root <none> and password file <none>
I20260807 05:13:18.358187 29355 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:18.358243 29355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:18.358407 29355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:18.360301 29355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-1/data/instance:
uuid: "cc4be873faa4452fa7f0f60372abf7de"
format_stamp: "Formatted at 2026-08-07 05:13:18 on dist-test-slave-49h3"
server_key: "15f980fdf7ec76e611b94aa19150e172"
server_key_iv: "dc4f14f8fe15a40d5040b8a39dadc1d3"
server_key_version: "encryptionkey@0"
I20260807 05:13:18.360797 29355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance:
uuid: "cc4be873faa4452fa7f0f60372abf7de"
format_stamp: "Formatted at 2026-08-07 05:13:18 on dist-test-slave-49h3"
server_key: "15f980fdf7ec76e611b94aa19150e172"
server_key_iv: "dc4f14f8fe15a40d5040b8a39dadc1d3"
server_key_version: "encryptionkey@0"
I20260807 05:13:18.364156 29355 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:13:18.366413 29369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:18.367499 29355 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:13:18.367591 29355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-1/wal
uuid: "cc4be873faa4452fa7f0f60372abf7de"
format_stamp: "Formatted at 2026-08-07 05:13:18 on dist-test-slave-49h3"
server_key: "15f980fdf7ec76e611b94aa19150e172"
server_key_iv: "dc4f14f8fe15a40d5040b8a39dadc1d3"
server_key_version: "encryptionkey@0"
I20260807 05:13:18.367684 29355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:18.388113 29355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:18.388797 29355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:18.388950 29355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:18.389572 29355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:13:18.390527 29355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:13:18.390574 29355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:18.390622 29355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:13:18.390643 29355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:18.399233 29355 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.66:35363
I20260807 05:13:18.399263 29490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.66:35363 every 8 connection(s)
I20260807 05:13:18.400205 29355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-1/data/info.pb
I20260807 05:13:18.400830 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 29355
I20260807 05:13:18.400951 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-1/wal/instance
I20260807 05:13:18.401237 8257 external_mini_cluster.cc:1428] Setting key 3fd3aad7ddc65ccc3b93608bbb7acb58
I20260807 05:13:18.403460 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.16.67:0
--local_ip_for_outbound_sockets=127.8.16.67
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36143
--builtin_ntp_servers=127.8.16.84:36095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 05:13:18.410020 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:18.402112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:47495 (local address 127.8.16.126:36143)
0807 05:13:18.402249 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:18.402254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:18.403389 (+ 1135us) server_negotiation.cc:407] Connection header received
0807 05:13:18.404788 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:18.404791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:18.404848 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:18.404924 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:18.405878 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.406429 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.407978 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.408245 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.408347 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:18.409574 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:18.409577 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:18.409579 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:18.409593 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:18.409614 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:18.409622 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:18.409684 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:18.409848 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 05:13:18.409900 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 05:13:18.411221 29494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:18.402549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36143 (local address 127.8.16.66:47495)
0807 05:13:18.403244 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:18.403287 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:18.404581 (+ 1294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:18.405105 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:18.405112 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:18.405198 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:18.405745 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:18.405759 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.406551 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.406554 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:18.407853 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:18.407864 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.408412 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.408417 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:18.408614 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:18.409253 (+ 639us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:13:18.409255 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:18.409313 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:13:18.409398 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:18.409702 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:18.409710 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:18.409725 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 05:13:18.409857 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 05:13:18.410164 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":133,"threads_started":1}
I20260807 05:13:18.412468 29491 heartbeater.cc:347] Connected to a master server at 127.8.16.126:36143
I20260807 05:13:18.412712 29491 heartbeater.cc:464] Registering TS with master...
I20260807 05:13:18.413237 29491 heartbeater.cc:511] Master 127.8.16.126:36143 requested a full tablet report, sending...
I20260807 05:13:18.414356 29154 ts_manager.cc:194] Registered new tserver with Master: cc4be873faa4452fa7f0f60372abf7de (127.8.16.66:35363)
I20260807 05:13:18.414935 29154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.16.66:47495
W20260807 05:13:18.513592 29495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:18.513844 29495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:18.513883 29495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:18.517813 29495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:18.517870 29495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:18.517922 29495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.16.67
I20260807 05:13:18.522545 29495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:36095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.16.67
--webserver_port=0
--tserver_master_addrs=127.8.16.126:36143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.16.67
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:18.523617 29495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:18.524976 29495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:18.534439 29501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:18.534440 29500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:18.534439 29503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:18.534672 29495 server_base.cc:1034] running on GCE node
I20260807 05:13:18.535162 29495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:18.535804 29495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:18.536969 29495 hybrid_clock.cc:648] HybridClock initialized: now 1786079598536947 us; error 31 us; skew 500 ppm
I20260807 05:13:18.538810 29495 webserver.cc:460] Webserver started at http://127.8.16.67:33551/ using document root <none> and password file <none>
I20260807 05:13:18.539323 29495 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:18.539381 29495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:18.539541 29495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:18.541533 29495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-2/data/instance:
uuid: "6763c9d242f14ead8b842581cfaba13c"
format_stamp: "Formatted at 2026-08-07 05:13:18 on dist-test-slave-49h3"
server_key: "cdf9341ca74dc05ab04b94051ea79fc8"
server_key_iv: "676137ac6a2c25a1092c715983fc62f1"
server_key_version: "encryptionkey@0"
I20260807 05:13:18.542059 29495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance:
uuid: "6763c9d242f14ead8b842581cfaba13c"
format_stamp: "Formatted at 2026-08-07 05:13:18 on dist-test-slave-49h3"
server_key: "cdf9341ca74dc05ab04b94051ea79fc8"
server_key_iv: "676137ac6a2c25a1092c715983fc62f1"
server_key_version: "encryptionkey@0"
I20260807 05:13:18.545768 29495 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260807 05:13:18.548079 29509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:18.549244 29495 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:13:18.549350 29495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-2/wal
uuid: "6763c9d242f14ead8b842581cfaba13c"
format_stamp: "Formatted at 2026-08-07 05:13:18 on dist-test-slave-49h3"
server_key: "cdf9341ca74dc05ab04b94051ea79fc8"
server_key_iv: "676137ac6a2c25a1092c715983fc62f1"
server_key_version: "encryptionkey@0"
I20260807 05:13:18.549438 29495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:18.571274 29495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:18.571947 29495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:18.572103 29495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:18.572716 29495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:13:18.573699 29495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:13:18.573750 29495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:18.573788 29495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:13:18.573807 29495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:18.582056 29495 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.67:39455
I20260807 05:13:18.582103 29629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.67:39455 every 8 connection(s)
I20260807 05:13:18.582935 29495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-2/data/info.pb
I20260807 05:13:18.591306 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 29495
I20260807 05:13:18.591415 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-2/wal/instance
I20260807 05:13:18.591727 8257 external_mini_cluster.cc:1428] Setting key e7d31e368d67ea709a61be2f348db5e2
I20260807 05:13:18.591905 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:18.584679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.67:53979 (local address 127.8.16.126:36143)
0807 05:13:18.584825 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:18.584828 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:18.585794 (+ 966us) server_negotiation.cc:407] Connection header received
0807 05:13:18.587148 (+ 1354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:18.587152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:18.587209 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:18.587298 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:18.588279 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.588824 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.590013 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.590244 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.590335 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:18.591363 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:18.591367 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:18.591370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:18.591383 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:18.591402 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:18.591410 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:18.591474 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:18.591697 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 05:13:18.591755 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 05:13:18.592892 29633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:18.584998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36143 (local address 127.8.16.67:53979)
0807 05:13:18.585648 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:18.585688 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:18.586936 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:18.587578 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:18.587584 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:18.587664 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:18.588134 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:18.588146 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.588946 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.588949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:18.589879 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:18.589889 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.590384 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.590388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:18.590552 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:18.591104 (+ 552us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:13:18.591105 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:18.591145 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:13:18.591217 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:18.591525 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:18.591535 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:18.591542 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 05:13:18.591698 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 05:13:18.591986 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":111,"threads_started":1}
I20260807 05:13:18.593964 29630 heartbeater.cc:347] Connected to a master server at 127.8.16.126:36143
I20260807 05:13:18.594201 29630 heartbeater.cc:464] Registering TS with master...
I20260807 05:13:18.594700 29630 heartbeater.cc:511] Master 127.8.16.126:36143 requested a full tablet report, sending...
I20260807 05:13:18.595681 29154 ts_manager.cc:194] Registered new tserver with Master: 6763c9d242f14ead8b842581cfaba13c (127.8.16.67:39455)
I20260807 05:13:18.596239 29154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.16.67:53979
I20260807 05:13:18.605512 8257 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:13:18.611164 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:18.607452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60944 (local address 127.8.16.126:36143)
0807 05:13:18.607571 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:18.607575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:18.607792 (+ 217us) server_negotiation.cc:407] Connection header received
0807 05:13:18.607903 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:18.607906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:18.607949 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:18.608012 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:18.608318 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.608827 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.609739 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.609949 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.610033 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:18.610777 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:18.610779 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:18.610782 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:18.610792 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:18.610810 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:18.610818 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:18.610876 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:18.610973 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 05:13:18.611026 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 05:13:18.613834 29154 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:13:18.616758 29154 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:13:18.630208 29645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:18.625011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55410 (local address 127.8.16.65:43723)
0807 05:13:18.625920 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:18.625935 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:18.625960 (+ 25us) server_negotiation.cc:407] Connection header received
0807 05:13:18.626037 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:18.626042 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:18.626277 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:18.626415 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:18.626674 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.627644 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.628983 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.629561 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.629654 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:18.629861 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:18.629961 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 05:13:18.630053 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":805,"thread_start_us":188,"threads_started":1}
I20260807 05:13:18.630201 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:18.624823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:43723 (local address 127.0.0.1:55410)
0807 05:13:18.625424 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:18.625510 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:18.625634 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:18.626410 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:18.626415 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:18.626435 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:18.626566 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:18.626573 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.627746 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.627749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:18.628870 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:18.628879 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.629666 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.629670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:18.629786 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:18.629823 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 05:13:18.629933 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 05:13:18.630051 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":97,"threads_started":1}
I20260807 05:13:18.630964 29643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:18.625077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:39455 (local address 127.0.0.1:46356)
0807 05:13:18.625531 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:18.625545 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:18.625642 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:18.627067 (+ 1425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:18.627075 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:18.627092 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:18.627168 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:18.627173 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.628447 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.628451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:18.629571 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:18.629579 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.630596 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.630600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:18.630685 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:18.630716 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 05:13:18.630775 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 05:13:18.630837 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":87,"threads_started":1}
I20260807 05:13:18.631073 29649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:18.626282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46356 (local address 127.8.16.67:39455)
0807 05:13:18.626682 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:18.626693 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:18.626715 (+ 22us) server_negotiation.cc:407] Connection header received
0807 05:13:18.626786 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:18.626790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:18.626950 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:18.627109 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:18.627249 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.628340 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.629708 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.630494 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.630592 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:18.630759 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:18.630808 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 05:13:18.630897 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":125,"threads_started":1}
I20260807 05:13:18.631695 29647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:18.625844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:35363 (local address 127.0.0.1:41586)
0807 05:13:18.626140 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:18.626152 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:18.626242 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:18.626784 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:18.626787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:18.626805 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:18.626893 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:18.626899 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.628302 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.628306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:18.629541 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:18.629553 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.631323 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.631327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:18.631414 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:18.631435 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 05:13:18.631505 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 05:13:18.631575 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":96,"threads_started":1}
I20260807 05:13:18.632156 29648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:18.625872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41586 (local address 127.8.16.66:35363)
0807 05:13:18.626337 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:18.626350 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:18.626375 (+ 25us) server_negotiation.cc:407] Connection header received
0807 05:13:18.626452 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:18.626457 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:18.626645 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:18.626790 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:18.627007 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.628169 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.629712 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.630978 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.631249 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:18.631419 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:18.631889 (+ 470us) server_negotiation.cc:299] Negotiation successful
0807 05:13:18.631987 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":130,"threads_started":1}
I20260807 05:13:18.633556 29560 tablet_service.cc:1467] Processing CreateTablet for tablet 89a6868a3e9c4b4a9e88e207207b3368 (DEFAULT_TABLE table=test-table [id=5675ab682b184849bac4cce74583de79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:13:18.633558 29281 tablet_service.cc:1467] Processing CreateTablet for tablet 89a6868a3e9c4b4a9e88e207207b3368 (DEFAULT_TABLE table=test-table [id=5675ab682b184849bac4cce74583de79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:13:18.634274 29420 tablet_service.cc:1467] Processing CreateTablet for tablet 89a6868a3e9c4b4a9e88e207207b3368 (DEFAULT_TABLE table=test-table [id=5675ab682b184849bac4cce74583de79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:13:18.634652 29281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89a6868a3e9c4b4a9e88e207207b3368. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:18.634652 29560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89a6868a3e9c4b4a9e88e207207b3368. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:18.635083 29420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89a6868a3e9c4b4a9e88e207207b3368. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:18.640945 29650 tablet_bootstrap.cc:492] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de: Bootstrap starting.
I20260807 05:13:18.641539 29652 tablet_bootstrap.cc:492] T 89a6868a3e9c4b4a9e88e207207b3368 P 6763c9d242f14ead8b842581cfaba13c: Bootstrap starting.
I20260807 05:13:18.643826 29650 tablet_bootstrap.cc:654] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:18.643826 29652 tablet_bootstrap.cc:654] T 89a6868a3e9c4b4a9e88e207207b3368 P 6763c9d242f14ead8b842581cfaba13c: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:18.644243 29653 tablet_bootstrap.cc:492] T 89a6868a3e9c4b4a9e88e207207b3368 P ccf491001b4d431da5bb20c3011b5c23: Bootstrap starting.
I20260807 05:13:18.644709 29650 log.cc:824] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:18.644709 29652 log.cc:824] T 89a6868a3e9c4b4a9e88e207207b3368 P 6763c9d242f14ead8b842581cfaba13c: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:18.646164 29653 tablet_bootstrap.cc:654] T 89a6868a3e9c4b4a9e88e207207b3368 P ccf491001b4d431da5bb20c3011b5c23: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:18.646735 29652 tablet_bootstrap.cc:492] T 89a6868a3e9c4b4a9e88e207207b3368 P 6763c9d242f14ead8b842581cfaba13c: No bootstrap required, opened a new log
I20260807 05:13:18.646906 29652 ts_tablet_manager.cc:1397] T 89a6868a3e9c4b4a9e88e207207b3368 P 6763c9d242f14ead8b842581cfaba13c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 05:13:18.647083 29653 log.cc:824] T 89a6868a3e9c4b4a9e88e207207b3368 P ccf491001b4d431da5bb20c3011b5c23: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:18.649022 29653 tablet_bootstrap.cc:492] T 89a6868a3e9c4b4a9e88e207207b3368 P ccf491001b4d431da5bb20c3011b5c23: No bootstrap required, opened a new log
I20260807 05:13:18.649156 29650 tablet_bootstrap.cc:492] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de: No bootstrap required, opened a new log
I20260807 05:13:18.649217 29653 ts_tablet_manager.cc:1397] T 89a6868a3e9c4b4a9e88e207207b3368 P ccf491001b4d431da5bb20c3011b5c23: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 05:13:18.649322 29650 ts_tablet_manager.cc:1397] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260807 05:13:18.650311 29652 raft_consensus.cc:348] T 89a6868a3e9c4b4a9e88e207207b3368 P 6763c9d242f14ead8b842581cfaba13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf491001b4d431da5bb20c3011b5c23" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 43723 } } peers { permanent_uuid: "6763c9d242f14ead8b842581cfaba13c" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 39455 } } peers { permanent_uuid: "cc4be873faa4452fa7f0f60372abf7de" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 35363 } }
I20260807 05:13:18.650529 29652 raft_consensus.cc:374] T 89a6868a3e9c4b4a9e88e207207b3368 P 6763c9d242f14ead8b842581cfaba13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:18.650590 29652 raft_consensus.cc:729] T 89a6868a3e9c4b4a9e88e207207b3368 P 6763c9d242f14ead8b842581cfaba13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6763c9d242f14ead8b842581cfaba13c, State: Initialized, Role: FOLLOWER
I20260807 05:13:18.651016 29652 consensus_queue.cc:260] T 89a6868a3e9c4b4a9e88e207207b3368 P 6763c9d242f14ead8b842581cfaba13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf491001b4d431da5bb20c3011b5c23" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 43723 } } peers { permanent_uuid: "6763c9d242f14ead8b842581cfaba13c" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 39455 } } peers { permanent_uuid: "cc4be873faa4452fa7f0f60372abf7de" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 35363 } }
I20260807 05:13:18.651666 29630 heartbeater.cc:503] Master 127.8.16.126:36143 was elected leader, sending a full tablet report...
I20260807 05:13:18.651947 29652 ts_tablet_manager.cc:1428] T 89a6868a3e9c4b4a9e88e207207b3368 P 6763c9d242f14ead8b842581cfaba13c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 05:13:18.653143 29653 raft_consensus.cc:348] T 89a6868a3e9c4b4a9e88e207207b3368 P ccf491001b4d431da5bb20c3011b5c23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf491001b4d431da5bb20c3011b5c23" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 43723 } } peers { permanent_uuid: "6763c9d242f14ead8b842581cfaba13c" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 39455 } } peers { permanent_uuid: "cc4be873faa4452fa7f0f60372abf7de" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 35363 } }
I20260807 05:13:18.653396 29653 raft_consensus.cc:374] T 89a6868a3e9c4b4a9e88e207207b3368 P ccf491001b4d431da5bb20c3011b5c23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:18.653481 29653 raft_consensus.cc:729] T 89a6868a3e9c4b4a9e88e207207b3368 P ccf491001b4d431da5bb20c3011b5c23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf491001b4d431da5bb20c3011b5c23, State: Initialized, Role: FOLLOWER
I20260807 05:13:18.653550 29650 raft_consensus.cc:348] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf491001b4d431da5bb20c3011b5c23" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 43723 } } peers { permanent_uuid: "6763c9d242f14ead8b842581cfaba13c" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 39455 } } peers { permanent_uuid: "cc4be873faa4452fa7f0f60372abf7de" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 35363 } }
I20260807 05:13:18.653798 29650 raft_consensus.cc:374] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:18.653855 29650 raft_consensus.cc:729] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4be873faa4452fa7f0f60372abf7de, State: Initialized, Role: FOLLOWER
I20260807 05:13:18.653913 29653 consensus_queue.cc:260] T 89a6868a3e9c4b4a9e88e207207b3368 P ccf491001b4d431da5bb20c3011b5c23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf491001b4d431da5bb20c3011b5c23" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 43723 } } peers { permanent_uuid: "6763c9d242f14ead8b842581cfaba13c" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 39455 } } peers { permanent_uuid: "cc4be873faa4452fa7f0f60372abf7de" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 35363 } }
I20260807 05:13:18.654251 29650 consensus_queue.cc:260] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf491001b4d431da5bb20c3011b5c23" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 43723 } } peers { permanent_uuid: "6763c9d242f14ead8b842581cfaba13c" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 39455 } } peers { permanent_uuid: "cc4be873faa4452fa7f0f60372abf7de" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 35363 } }
I20260807 05:13:18.654758 29352 heartbeater.cc:503] Master 127.8.16.126:36143 was elected leader, sending a full tablet report...
I20260807 05:13:18.655001 29653 ts_tablet_manager.cc:1428] T 89a6868a3e9c4b4a9e88e207207b3368 P ccf491001b4d431da5bb20c3011b5c23: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 05:13:18.655258 29650 ts_tablet_manager.cc:1428] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260807 05:13:18.655601 29491 heartbeater.cc:503] Master 127.8.16.126:36143 was elected leader, sending a full tablet report...
W20260807 05:13:18.719470 29353 tablet.cc:2369] T 89a6868a3e9c4b4a9e88e207207b3368 P ccf491001b4d431da5bb20c3011b5c23: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:13:18.752642 29659 raft_consensus.cc:482] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:13:18.752873 29659 raft_consensus.cc:504] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf491001b4d431da5bb20c3011b5c23" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 43723 } } peers { permanent_uuid: "6763c9d242f14ead8b842581cfaba13c" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 39455 } } peers { permanent_uuid: "cc4be873faa4452fa7f0f60372abf7de" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 35363 } }
I20260807 05:13:18.754160 29659 leader_election.cc:288] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf491001b4d431da5bb20c3011b5c23 (127.8.16.65:43723), 6763c9d242f14ead8b842581cfaba13c (127.8.16.67:39455)
I20260807 05:13:18.758481 29649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:18.754500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:59899 (local address 127.8.16.67:39455)
0807 05:13:18.754624 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:18.754629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:18.754916 (+ 287us) server_negotiation.cc:407] Connection header received
0807 05:13:18.755029 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:18.755031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:18.755070 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:18.755129 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:18.755388 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.755946 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.757249 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.757859 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.757966 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:18.758028 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:18.758293 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 05:13:18.758361 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 05:13:18.758481 29645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:18.754522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.16.66:40425 (local address 127.8.16.65:43723)
0807 05:13:18.754661 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:18.754665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:18.754684 (+ 19us) server_negotiation.cc:407] Connection header received
0807 05:13:18.754809 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:18.754812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:18.754855 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:18.754929 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:13:18.755198 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.755748 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.757217 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.757859 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.757966 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:18.758029 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:18.758278 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 05:13:18.758349 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 05:13:18.758496 29494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:18.754374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:43723 (local address 127.8.16.66:40425)
0807 05:13:18.754575 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:18.754589 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:18.754704 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:18.754958 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:18.754961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:18.754979 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:18.755090 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:18.755095 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.755848 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.755851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:18.757079 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:18.757089 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.757972 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.757975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:18.758110 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:18.758174 (+ 64us) client_negotiation.cc:769] Sending connection context
0807 05:13:18.758254 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 05:13:18.758335 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":42}
I20260807 05:13:18.758504 29660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:18.754433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.67:39455 (local address 127.8.16.66:59899)
0807 05:13:18.754830 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:18.754841 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:18.754931 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:18.755154 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:18.755157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:18.755171 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:13:18.755284 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:18.755289 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.756060 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.756063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:18.757147 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:18.757155 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.757967 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.757971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:18.758130 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:18.758184 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 05:13:18.758254 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 05:13:18.758335 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":82,"thread_start_us":111,"threads_started":1}
I20260807 05:13:18.759189 29303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a6868a3e9c4b4a9e88e207207b3368" candidate_uuid: "cc4be873faa4452fa7f0f60372abf7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf491001b4d431da5bb20c3011b5c23" is_pre_election: true
I20260807 05:13:18.759523 29303 raft_consensus.cc:2454] T 89a6868a3e9c4b4a9e88e207207b3368 P ccf491001b4d431da5bb20c3011b5c23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc4be873faa4452fa7f0f60372abf7de in term 0.
I20260807 05:13:18.759552 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a6868a3e9c4b4a9e88e207207b3368" candidate_uuid: "cc4be873faa4452fa7f0f60372abf7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6763c9d242f14ead8b842581cfaba13c" is_pre_election: true
I20260807 05:13:18.759799 29580 raft_consensus.cc:2454] T 89a6868a3e9c4b4a9e88e207207b3368 P 6763c9d242f14ead8b842581cfaba13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc4be873faa4452fa7f0f60372abf7de in term 0.
I20260807 05:13:18.760033 29374 leader_election.cc:302] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc4be873faa4452fa7f0f60372abf7de, ccf491001b4d431da5bb20c3011b5c23; no voters:
I20260807 05:13:18.760329 29659 raft_consensus.cc:2781] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:13:18.760411 29659 raft_consensus.cc:482] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:13:18.760458 29659 raft_consensus.cc:3037] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:18.761490 29659 raft_consensus.cc:504] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf491001b4d431da5bb20c3011b5c23" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 43723 } } peers { permanent_uuid: "6763c9d242f14ead8b842581cfaba13c" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 39455 } } peers { permanent_uuid: "cc4be873faa4452fa7f0f60372abf7de" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 35363 } }
I20260807 05:13:18.761797 29659 leader_election.cc:288] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de [CANDIDATE]: Term 1 election: Requested vote from peers ccf491001b4d431da5bb20c3011b5c23 (127.8.16.65:43723), 6763c9d242f14ead8b842581cfaba13c (127.8.16.67:39455)
I20260807 05:13:18.762094 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a6868a3e9c4b4a9e88e207207b3368" candidate_uuid: "cc4be873faa4452fa7f0f60372abf7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6763c9d242f14ead8b842581cfaba13c"
I20260807 05:13:18.762102 29303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a6868a3e9c4b4a9e88e207207b3368" candidate_uuid: "cc4be873faa4452fa7f0f60372abf7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf491001b4d431da5bb20c3011b5c23"
I20260807 05:13:18.762207 29580 raft_consensus.cc:3037] T 89a6868a3e9c4b4a9e88e207207b3368 P 6763c9d242f14ead8b842581cfaba13c [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:18.762221 29303 raft_consensus.cc:3037] T 89a6868a3e9c4b4a9e88e207207b3368 P ccf491001b4d431da5bb20c3011b5c23 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:18.763175 29303 raft_consensus.cc:2454] T 89a6868a3e9c4b4a9e88e207207b3368 P ccf491001b4d431da5bb20c3011b5c23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc4be873faa4452fa7f0f60372abf7de in term 1.
I20260807 05:13:18.763207 29580 raft_consensus.cc:2454] T 89a6868a3e9c4b4a9e88e207207b3368 P 6763c9d242f14ead8b842581cfaba13c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc4be873faa4452fa7f0f60372abf7de in term 1.
I20260807 05:13:18.763465 29374 leader_election.cc:302] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc4be873faa4452fa7f0f60372abf7de, ccf491001b4d431da5bb20c3011b5c23; no voters:
I20260807 05:13:18.763634 29659 raft_consensus.cc:2781] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:18.763828 29659 raft_consensus.cc:686] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de [term 1 LEADER]: Becoming Leader. State: Replica: cc4be873faa4452fa7f0f60372abf7de, State: Running, Role: LEADER
I20260807 05:13:18.764153 29659 consensus_queue.cc:237] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf491001b4d431da5bb20c3011b5c23" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 43723 } } peers { permanent_uuid: "6763c9d242f14ead8b842581cfaba13c" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 39455 } } peers { permanent_uuid: "cc4be873faa4452fa7f0f60372abf7de" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 35363 } }
I20260807 05:13:18.767244 29154 catalog_manager.cc:5495] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de reported cstate change: term changed from 0 to 1, leader changed from <none> to cc4be873faa4452fa7f0f60372abf7de (127.8.16.66). New cstate: current_term: 1 leader_uuid: "cc4be873faa4452fa7f0f60372abf7de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf491001b4d431da5bb20c3011b5c23" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 43723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6763c9d242f14ead8b842581cfaba13c" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 39455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc4be873faa4452fa7f0f60372abf7de" member_type: VOTER last_known_addr { host: "127.8.16.66" port: 35363 } health_report { overall_health: HEALTHY } } }
I20260807 05:13:18.781301 29648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:18.778134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41600 (local address 127.8.16.66:35363)
0807 05:13:18.778237 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:18.778240 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:18.778252 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:13:18.778360 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:18.778362 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:18.778412 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:18.778496 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:13:18.778682 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.779165 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.779997 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:18.780194 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:18.780260 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:18.780478 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:13:18.780895 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:13:18.780984 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:18.781079 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 05:13:18.781123 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 05:13:18.786725 29580 raft_consensus.cc:1261] T 89a6868a3e9c4b4a9e88e207207b3368 P 6763c9d242f14ead8b842581cfaba13c [term 1 FOLLOWER]: Refusing update from remote peer cc4be873faa4452fa7f0f60372abf7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:13:18.786736 29303 raft_consensus.cc:1261] T 89a6868a3e9c4b4a9e88e207207b3368 P ccf491001b4d431da5bb20c3011b5c23 [term 1 FOLLOWER]: Refusing update from remote peer cc4be873faa4452fa7f0f60372abf7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:13:18.787432 29661 consensus_queue.cc:1035] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "6763c9d242f14ead8b842581cfaba13c" member_type: VOTER last_known_addr { host: "127.8.16.67" port: 39455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:13:18.787601 29659 consensus_queue.cc:1035] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf491001b4d431da5bb20c3011b5c23" member_type: VOTER last_known_addr { host: "127.8.16.65" port: 43723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:13:18.794837 29667 mvcc.cc:205] Tried to move back new op lower bound from 7315782036623437824 to 7315782036540706816. Current Snapshot: MvccSnapshot[applied={T|T < 7315782036623437824}]
I20260807 05:13:18.795080 29665 mvcc.cc:205] Tried to move back new op lower bound from 7315782036623437824 to 7315782036540706816. Current Snapshot: MvccSnapshot[applied={T|T < 7315782036623437824}]
I20260807 05:13:18.802708 29154 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:13:18.802865 29154 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:13:18.805083 29154 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5feb561d53eb4a77a3630ce831a543e3: Sending DeleteTablet for 3 replicas of tablet 89a6868a3e9c4b4a9e88e207207b3368
I20260807 05:13:18.805866 29420 tablet_service.cc:1514] Processing DeleteTablet for tablet 89a6868a3e9c4b4a9e88e207207b3368 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:13:18 UTC) from {username='slave'} at 127.0.0.1:41586
I20260807 05:13:18.805856 29281 tablet_service.cc:1514] Processing DeleteTablet for tablet 89a6868a3e9c4b4a9e88e207207b3368 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:13:18 UTC) from {username='slave'} at 127.0.0.1:55410
I20260807 05:13:18.805883 29560 tablet_service.cc:1514] Processing DeleteTablet for tablet 89a6868a3e9c4b4a9e88e207207b3368 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:13:18 UTC) from {username='slave'} at 127.0.0.1:46356
I20260807 05:13:18.806445 29675 tablet_replica.cc:330] T 89a6868a3e9c4b4a9e88e207207b3368 P 6763c9d242f14ead8b842581cfaba13c: stopping tablet replica
I20260807 05:13:18.807031 29675 raft_consensus.cc:2229] T 89a6868a3e9c4b4a9e88e207207b3368 P 6763c9d242f14ead8b842581cfaba13c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:13:18.807134 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 29217
I20260807 05:13:18.810045 29675 raft_consensus.cc:2258] T 89a6868a3e9c4b4a9e88e207207b3368 P 6763c9d242f14ead8b842581cfaba13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:13:18.811331 29675 ts_tablet_manager.cc:1905] T 89a6868a3e9c4b4a9e88e207207b3368 P 6763c9d242f14ead8b842581cfaba13c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:13:18.813517 29675 ts_tablet_manager.cc:1918] T 89a6868a3e9c4b4a9e88e207207b3368 P 6763c9d242f14ead8b842581cfaba13c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:13:18.813619 29675 log.cc:1196] T 89a6868a3e9c4b4a9e88e207207b3368 P 6763c9d242f14ead8b842581cfaba13c: Deleting WAL directory at /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079354071073-8257-0/minicluster-data/ts-2/wal/wals/89a6868a3e9c4b4a9e88e207207b3368
I20260807 05:13:18.813831 29678 tablet_replica.cc:330] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de: stopping tablet replica
I20260807 05:13:18.813948 29675 ts_tablet_manager.cc:1939] T 89a6868a3e9c4b4a9e88e207207b3368 P 6763c9d242f14ead8b842581cfaba13c: Deleting consensus metadata
I20260807 05:13:18.814260 29678 raft_consensus.cc:2229] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:13:18.814719 29678 raft_consensus.cc:2258] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:13:18.815284 29139 catalog_manager.cc:4841] TS 6763c9d242f14ead8b842581cfaba13c (127.8.16.67:39455): tablet 89a6868a3e9c4b4a9e88e207207b3368 (table test-table [id=5675ab682b184849bac4cce74583de79]) successfully deleted
W20260807 05:13:18.816069 29140 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.16.65:43723 (error 108)
I20260807 05:13:18.816370 29678 ts_tablet_manager.cc:1905] T 89a6868a3e9c4b4a9e88e207207b3368 P cc4be873faa4452fa7f0f60372abf7de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:13:18.816886 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 29355
I20260807 05:13:18.817087 29647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:18.816801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.65:43723 (local address 127.0.0.1:55416)
0807 05:13:18.816931 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:18.817005 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.65:43723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 05:13:18.817253 29140 catalog_manager.cc:4584] TS ccf491001b4d431da5bb20c3011b5c23 (127.8.16.65:43723): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 89a6868a3e9c4b4a9e88e207207b3368: Network error: Client connection negotiation failed: client connection to 127.8.16.65:43723: connect: Connection refused (error 111)
I20260807 05:13:18.822710 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 29495
I20260807 05:13:18.823500 29647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:18.822900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.66:35363 (local address 127.0.0.1:41610)
0807 05:13:18.823368 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:18.823417 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.66:35363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390}
I20260807 05:13:18.830397 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 29120
2026-08-07T05:13:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1052 ms)
[----------] 6 tests from EncryptionPolicyTest (9853 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:13:18 dist-test-slave-49h3 krb5kdc[29682](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:13:18 dist-test-slave-49h3 krb5kdc[29682](info): set up 2 sockets
Aug 07 05:13:18 dist-test-slave-49h3 krb5kdc[29682](info): commencing operation
krb5kdc: starting...
W20260807 05:13:19.809028 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:13:19 dist-test-slave-49h3 krb5kdc[29682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:13:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:13:19.926590 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34409
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:20.040148 29697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:20.040545 29697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:20.040623 29697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:20.044811 29697 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:20.044883 29697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:20.044901 29697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:20.044916 29697 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:20.044930 29697 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:20.050400 29697 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34409
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:20.051517 29697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:20.052572 29697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:20.060830 29705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:20.060972 29702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:20.060854 29703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:20.060869 29697 server_base.cc:1034] running on GCE node
I20260807 05:13:20.061657 29697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:20.062713 29697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:20.063880 29697 hybrid_clock.cc:648] HybridClock initialized: now 1786079600063861 us; error 31 us; skew 500 ppm
Aug 07 05:13:20 dist-test-slave-49h3 krb5kdc[29682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079600, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:20.070611 29697 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:13:20.071521 29697 webserver.cc:460] Webserver started at http://127.0.0.1:35829/ using document root <none> and password file <none>
I20260807 05:13:20.072037 29697 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:20.072096 29697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:20.072296 29697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:20.074345 29697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "06b2fc7abd7c424b84f8b1e945243056"
format_stamp: "Formatted at 2026-08-07 05:13:20 on dist-test-slave-49h3"
server_key: "ed9f0988ed92aef4827dfad9e21876e9"
server_key_iv: "c1e9a31529cc7755247c372400fce430"
server_key_version: "encryptionkey@0"
I20260807 05:13:20.074863 29697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "06b2fc7abd7c424b84f8b1e945243056"
format_stamp: "Formatted at 2026-08-07 05:13:20 on dist-test-slave-49h3"
server_key: "ed9f0988ed92aef4827dfad9e21876e9"
server_key_iv: "c1e9a31529cc7755247c372400fce430"
server_key_version: "encryptionkey@0"
I20260807 05:13:20.078390 29697 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:13:20.080699 29712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:20.081908 29697 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:13:20.082016 29697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "06b2fc7abd7c424b84f8b1e945243056"
format_stamp: "Formatted at 2026-08-07 05:13:20 on dist-test-slave-49h3"
server_key: "ed9f0988ed92aef4827dfad9e21876e9"
server_key_iv: "c1e9a31529cc7755247c372400fce430"
server_key_version: "encryptionkey@0"
I20260807 05:13:20.082119 29697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:20.126577 29697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:20.129650 29697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:20.129849 29697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:20.136868 29697 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34409
I20260807 05:13:20.136883 29770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34409 every 8 connection(s)
I20260807 05:13:20.137840 29697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:20.140499 29771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:20.143563 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 29697
I20260807 05:13:20.143683 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:20.143970 8257 external_mini_cluster.cc:1428] Setting key c7b523a2c7b884dea857d0f3c8325cc3
I20260807 05:13:20.146327 29771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06b2fc7abd7c424b84f8b1e945243056: Bootstrap starting.
I20260807 05:13:20.148940 29771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06b2fc7abd7c424b84f8b1e945243056: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:20.149807 29771 log.cc:824] T 00000000000000000000000000000000 P 06b2fc7abd7c424b84f8b1e945243056: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:20.151942 29771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06b2fc7abd7c424b84f8b1e945243056: No bootstrap required, opened a new log
Aug 07 05:13:20 dist-test-slave-49h3 krb5kdc[29682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 05:13:20.156019 29771 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06b2fc7abd7c424b84f8b1e945243056 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06b2fc7abd7c424b84f8b1e945243056" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34409 } }
I20260807 05:13:20.156289 29771 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06b2fc7abd7c424b84f8b1e945243056 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:20.156402 29771 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06b2fc7abd7c424b84f8b1e945243056 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06b2fc7abd7c424b84f8b1e945243056, State: Initialized, Role: FOLLOWER
I20260807 05:13:20.156864 29771 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06b2fc7abd7c424b84f8b1e945243056 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06b2fc7abd7c424b84f8b1e945243056" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34409 } }
I20260807 05:13:20.157028 29771 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06b2fc7abd7c424b84f8b1e945243056 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:20.157102 29771 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06b2fc7abd7c424b84f8b1e945243056 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:20.157197 29771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06b2fc7abd7c424b84f8b1e945243056 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:20.158329 29771 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06b2fc7abd7c424b84f8b1e945243056 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06b2fc7abd7c424b84f8b1e945243056" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34409 } }
I20260807 05:13:20.158697 29771 leader_election.cc:302] T 00000000000000000000000000000000 P 06b2fc7abd7c424b84f8b1e945243056 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06b2fc7abd7c424b84f8b1e945243056; no voters:
I20260807 05:13:20.159036 29771 leader_election.cc:288] T 00000000000000000000000000000000 P 06b2fc7abd7c424b84f8b1e945243056 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:20.159153 29778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06b2fc7abd7c424b84f8b1e945243056 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:20.159353 29778 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06b2fc7abd7c424b84f8b1e945243056 [term 1 LEADER]: Becoming Leader. State: Replica: 06b2fc7abd7c424b84f8b1e945243056, State: Running, Role: LEADER
I20260807 05:13:20.159566 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:20.145390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45263 (local address 127.0.0.1:34409)
0807 05:13:20.145846 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:20.145865 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:20.145909 (+ 44us) server_negotiation.cc:407] Connection header received
0807 05:13:20.146730 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:20.146754 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:20.147176 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:20.147543 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:20.148226 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:20.149736 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:20.150776 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:20.151443 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:20.151590 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:20.154265 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:20.154271 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:20.154287 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:20.154321 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:20.156237 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:20.156784 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:20.156790 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:20.156795 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:20.156936 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:20.157297 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:20.157300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:20.157302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:20.157817 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:20.158042 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:20.158358 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 05:13:20.158586 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":114,"threads_started":1}
I20260807 05:13:20.159713 29778 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06b2fc7abd7c424b84f8b1e945243056 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06b2fc7abd7c424b84f8b1e945243056" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34409 } }
I20260807 05:13:20.160257 29771 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06b2fc7abd7c424b84f8b1e945243056 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:20.161218 29779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06b2fc7abd7c424b84f8b1e945243056 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06b2fc7abd7c424b84f8b1e945243056" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06b2fc7abd7c424b84f8b1e945243056" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34409 } } }
I20260807 05:13:20.161351 29779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06b2fc7abd7c424b84f8b1e945243056 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:20.161847 29787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:20.162146 29780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06b2fc7abd7c424b84f8b1e945243056 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06b2fc7abd7c424b84f8b1e945243056. Latest consensus state: current_term: 1 leader_uuid: "06b2fc7abd7c424b84f8b1e945243056" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06b2fc7abd7c424b84f8b1e945243056" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34409 } } }
I20260807 05:13:20.162318 29780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06b2fc7abd7c424b84f8b1e945243056 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:20.163507 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:13:20.164855 29787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:20.170706 29787 catalog_manager.cc:1347] Generated new cluster ID: 55049678cdcb42788af116b7298a92b4
I20260807 05:13:20.170770 29787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:20.174345 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:20.165729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58665 (local address 127.0.0.1:34409)
0807 05:13:20.165838 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:20.165843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:20.166087 (+ 244us) server_negotiation.cc:407] Connection header received
0807 05:13:20.166227 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:20.166230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:20.166274 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:20.166350 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:20.167254 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:20.167999 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:20.169164 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:20.169424 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:20.169549 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:20.171002 (+ 1453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:20.171006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:20.171008 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:20.171032 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:20.172586 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:20.173117 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:20.173120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:20.173122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:20.173197 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:20.173547 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:20.173549 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:20.173551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:20.173729 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:20.173846 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:20.174118 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 05:13:20.174205 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 05:13:20.199815 29787 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:20.200721 29787 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:20.208222 29787 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06b2fc7abd7c424b84f8b1e945243056: Generated new TSK 0
I20260807 05:13:20.208856 29787 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:13:20.216413 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 29697
W20260807 05:13:20.222769 29680 connection.cc:535] client connection to 127.0.0.1:34409 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34409): Connection reset by peer (error 104)
2026-08-07T05:13:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1384 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:13:20 dist-test-slave-49h3 krb5kdc[29805](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:13:20 dist-test-slave-49h3 krb5kdc[29805](info): set up 2 sockets
Aug 07 05:13:20 dist-test-slave-49h3 krb5kdc[29805](info): commencing operation
krb5kdc: starting...
W20260807 05:13:21.205091 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.969s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:13:21 dist-test-slave-49h3 krb5kdc[29805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:13:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:13:21.324086 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38695
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:21.432600 29820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:21.432859 29820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:21.432902 29820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:21.436937 29820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:21.436998 29820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:21.437017 29820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:21.437047 29820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:21.437074 29820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:21.442273 29820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38695
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:21.443281 29820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:21.444248 29820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:21.452369 29826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:21.452390 29827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:21.452381 29829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:21.452440 29820 server_base.cc:1034] running on GCE node
I20260807 05:13:21.453109 29820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:21.454227 29820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:21.455403 29820 hybrid_clock.cc:648] HybridClock initialized: now 1786079601455384 us; error 29 us; skew 500 ppm
Aug 07 05:13:21 dist-test-slave-49h3 krb5kdc[29805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079601, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:21.458221 29820 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:13:21.459116 29820 webserver.cc:460] Webserver started at http://127.0.0.1:39645/ using document root <none> and password file <none>
I20260807 05:13:21.459653 29820 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:21.459722 29820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:21.459929 29820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:21.461966 29820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "4f9aa7ceb34b4c4a8dfcca8cca4bd68e"
format_stamp: "Formatted at 2026-08-07 05:13:21 on dist-test-slave-49h3"
server_key: "aee36a99e7c5918b1360e8d27b71a75d"
server_key_iv: "93822afa76192bb1371d5b349ba14ee5"
server_key_version: "encryptionkey@0"
I20260807 05:13:21.462514 29820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "4f9aa7ceb34b4c4a8dfcca8cca4bd68e"
format_stamp: "Formatted at 2026-08-07 05:13:21 on dist-test-slave-49h3"
server_key: "aee36a99e7c5918b1360e8d27b71a75d"
server_key_iv: "93822afa76192bb1371d5b349ba14ee5"
server_key_version: "encryptionkey@0"
I20260807 05:13:21.465996 29820 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:13:21.468253 29837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:21.469405 29820 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:13:21.469535 29820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "4f9aa7ceb34b4c4a8dfcca8cca4bd68e"
format_stamp: "Formatted at 2026-08-07 05:13:21 on dist-test-slave-49h3"
server_key: "aee36a99e7c5918b1360e8d27b71a75d"
server_key_iv: "93822afa76192bb1371d5b349ba14ee5"
server_key_version: "encryptionkey@0"
I20260807 05:13:21.469635 29820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:21.494841 29820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:21.497927 29820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:21.498108 29820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:21.505081 29820 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38695
I20260807 05:13:21.505097 29891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38695 every 8 connection(s)
I20260807 05:13:21.506024 29820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:21.508679 29892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:21.509977 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 29820
I20260807 05:13:21.510113 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:21.510391 8257 external_mini_cluster.cc:1428] Setting key 84c940b3cdefbba1394ac2f8515b8d77
I20260807 05:13:21.514655 29892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f9aa7ceb34b4c4a8dfcca8cca4bd68e: Bootstrap starting.
I20260807 05:13:21.517210 29892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f9aa7ceb34b4c4a8dfcca8cca4bd68e: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:21.517993 29892 log.cc:824] T 00000000000000000000000000000000 P 4f9aa7ceb34b4c4a8dfcca8cca4bd68e: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:21.519098 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:21.511665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49067 (local address 127.0.0.1:38695)
0807 05:13:21.512079 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:21.512094 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:21.512129 (+ 35us) server_negotiation.cc:407] Connection header received
0807 05:13:21.512757 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:21.512774 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:21.513101 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:21.513380 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:21.513798 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:21.514755 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:21.515927 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:21.516390 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:21.516497 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:21.517399 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:21.517404 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:21.517417 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:21.517823 (+ 406us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:21.517878 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:21.517900 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:21.518001 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:21.518254 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 05:13:21.518395 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":107,"threads_started":1}
I20260807 05:13:21.519898 29892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f9aa7ceb34b4c4a8dfcca8cca4bd68e: No bootstrap required, opened a new log
I20260807 05:13:21.521957 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:13:21.523537 29892 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f9aa7ceb34b4c4a8dfcca8cca4bd68e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9aa7ceb34b4c4a8dfcca8cca4bd68e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } }
I20260807 05:13:21.523820 29892 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f9aa7ceb34b4c4a8dfcca8cca4bd68e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:21.523914 29892 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f9aa7ceb34b4c4a8dfcca8cca4bd68e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f9aa7ceb34b4c4a8dfcca8cca4bd68e, State: Initialized, Role: FOLLOWER
I20260807 05:13:21.524348 29892 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f9aa7ceb34b4c4a8dfcca8cca4bd68e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9aa7ceb34b4c4a8dfcca8cca4bd68e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } }
I20260807 05:13:21.524470 29892 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f9aa7ceb34b4c4a8dfcca8cca4bd68e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:21.524528 29892 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f9aa7ceb34b4c4a8dfcca8cca4bd68e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:21.524627 29892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f9aa7ceb34b4c4a8dfcca8cca4bd68e [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:21.525542 29892 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f9aa7ceb34b4c4a8dfcca8cca4bd68e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9aa7ceb34b4c4a8dfcca8cca4bd68e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } }
I20260807 05:13:21.525856 29892 leader_election.cc:302] T 00000000000000000000000000000000 P 4f9aa7ceb34b4c4a8dfcca8cca4bd68e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f9aa7ceb34b4c4a8dfcca8cca4bd68e; no voters:
I20260807 05:13:21.526172 29892 leader_election.cc:288] T 00000000000000000000000000000000 P 4f9aa7ceb34b4c4a8dfcca8cca4bd68e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:21.526278 29903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f9aa7ceb34b4c4a8dfcca8cca4bd68e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:21.526496 29903 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f9aa7ceb34b4c4a8dfcca8cca4bd68e [term 1 LEADER]: Becoming Leader. State: Replica: 4f9aa7ceb34b4c4a8dfcca8cca4bd68e, State: Running, Role: LEADER
I20260807 05:13:21.526837 29903 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f9aa7ceb34b4c4a8dfcca8cca4bd68e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9aa7ceb34b4c4a8dfcca8cca4bd68e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } }
I20260807 05:13:21.527455 29892 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f9aa7ceb34b4c4a8dfcca8cca4bd68e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:21.527923 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:21.523824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58631 (local address 127.0.0.1:38695)
0807 05:13:21.523943 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:21.523948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:21.524069 (+ 121us) server_negotiation.cc:407] Connection header received
0807 05:13:21.524191 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:21.524194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:21.524232 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:21.524301 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:21.524609 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:21.525145 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:21.526036 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:21.526419 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:21.526520 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:21.527466 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:21.527470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:21.527472 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:21.527486 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:21.527514 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:21.527526 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:21.527627 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:21.527759 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 05:13:21.527796 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 05:13:21.528769 29905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f9aa7ceb34b4c4a8dfcca8cca4bd68e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f9aa7ceb34b4c4a8dfcca8cca4bd68e. Latest consensus state: current_term: 1 leader_uuid: "4f9aa7ceb34b4c4a8dfcca8cca4bd68e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9aa7ceb34b4c4a8dfcca8cca4bd68e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } } }
I20260807 05:13:21.528916 29905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f9aa7ceb34b4c4a8dfcca8cca4bd68e [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:21.528769 29904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f9aa7ceb34b4c4a8dfcca8cca4bd68e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f9aa7ceb34b4c4a8dfcca8cca4bd68e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9aa7ceb34b4c4a8dfcca8cca4bd68e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } } }
I20260807 05:13:21.529109 29904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f9aa7ceb34b4c4a8dfcca8cca4bd68e [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:21.529368 29916 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:21.532207 29916 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:21.537237 29916 catalog_manager.cc:1347] Generated new cluster ID: 49ea92ef314f4d0c97b8b7c625cbb90f
I20260807 05:13:21.537307 29916 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:21.549285 29916 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:21.550520 29916 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:21.559113 29916 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f9aa7ceb34b4c4a8dfcca8cca4bd68e: Generated new TSK 0
I20260807 05:13:21.559844 29916 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:13:21.565196 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 29820
W20260807 05:13:21.571452 29803 connection.cc:535] client connection to 127.0.0.1:38695 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38695): Connection reset by peer (error 104)
2026-08-07T05:13:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1348 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:13:21 dist-test-slave-49h3 krb5kdc[29925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:13:21 dist-test-slave-49h3 krb5kdc[29925](info): set up 2 sockets
Aug 07 05:13:21 dist-test-slave-49h3 krb5kdc[29925](info): commencing operation
krb5kdc: starting...
W20260807 05:13:22.549585 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.964s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:13:22 dist-test-slave-49h3 krb5kdc[29925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:13:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:13:22.666119 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46585
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:22.774513 29938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:22.774791 29938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:22.774849 29938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:22.778831 29938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:22.778895 29938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:22.778923 29938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:22.778944 29938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:22.778972 29938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:22.784125 29938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46585
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:22.785133 29938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:22.786154 29938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:22.794214 29944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:22.794245 29947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:22.794350 29938 server_base.cc:1034] running on GCE node
W20260807 05:13:22.794212 29945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:22.794994 29938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:22.796029 29938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:22.797190 29938 hybrid_clock.cc:648] HybridClock initialized: now 1786079602797178 us; error 35 us; skew 500 ppm
Aug 07 05:13:22 dist-test-slave-49h3 krb5kdc[29925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079602, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:22.799863 29938 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:13:22.800704 29938 webserver.cc:460] Webserver started at http://127.0.0.1:43629/ using document root <none> and password file <none>
I20260807 05:13:22.801210 29938 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:22.801270 29938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:22.801436 29938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:22.803522 29938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "864699ced19e4dcba8bac1aa4a943181"
format_stamp: "Formatted at 2026-08-07 05:13:22 on dist-test-slave-49h3"
server_key: "089650039ddeec5831f741889ea3f4b8"
server_key_iv: "fda8b26d93b1d5c6d90b63688041638b"
server_key_version: "encryptionkey@0"
I20260807 05:13:22.804018 29938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "864699ced19e4dcba8bac1aa4a943181"
format_stamp: "Formatted at 2026-08-07 05:13:22 on dist-test-slave-49h3"
server_key: "089650039ddeec5831f741889ea3f4b8"
server_key_iv: "fda8b26d93b1d5c6d90b63688041638b"
server_key_version: "encryptionkey@0"
I20260807 05:13:22.807600 29938 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 05:13:22.809849 29955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:22.810951 29938 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:13:22.811062 29938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "864699ced19e4dcba8bac1aa4a943181"
format_stamp: "Formatted at 2026-08-07 05:13:22 on dist-test-slave-49h3"
server_key: "089650039ddeec5831f741889ea3f4b8"
server_key_iv: "fda8b26d93b1d5c6d90b63688041638b"
server_key_version: "encryptionkey@0"
I20260807 05:13:22.811156 29938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:22.815480 29938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:22.818151 29938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:22.818315 29938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:22.824764 29938 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46585
I20260807 05:13:22.824770 30010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46585 every 8 connection(s)
I20260807 05:13:22.825695 29938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:22.828284 30011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:22.831763 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 29938
I20260807 05:13:22.831900 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:22.832211 8257 external_mini_cluster.cc:1428] Setting key 22bc7a29b7f4c6721bdd6ba2b489de92
I20260807 05:13:22.834079 30011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 864699ced19e4dcba8bac1aa4a943181: Bootstrap starting.
I20260807 05:13:22.836422 30011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 864699ced19e4dcba8bac1aa4a943181: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:22.837111 30014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:22.833551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57617 (local address 127.0.0.1:46585)
0807 05:13:22.833941 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:22.833954 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:22.833990 (+ 36us) server_negotiation.cc:407] Connection header received
0807 05:13:22.834612 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:22.834630 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:22.834908 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:22.835176 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:22.835978 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:22.835982 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:22.835993 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:22.836003 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:22.836057 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:22.836075 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:22.836157 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:22.836277 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 05:13:22.836413 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":107,"threads_started":1}
I20260807 05:13:22.837144 30011 log.cc:824] T 00000000000000000000000000000000 P 864699ced19e4dcba8bac1aa4a943181: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:22.839118 30011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 864699ced19e4dcba8bac1aa4a943181: No bootstrap required, opened a new log
I20260807 05:13:22.839732 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:13:22.843029 30011 raft_consensus.cc:348] T 00000000000000000000000000000000 P 864699ced19e4dcba8bac1aa4a943181 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864699ced19e4dcba8bac1aa4a943181" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46585 } }
I20260807 05:13:22.843266 30014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:22.841774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38241 (local address 127.0.0.1:46585)
0807 05:13:22.841886 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:22.841890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:22.842057 (+ 167us) server_negotiation.cc:407] Connection header received
0807 05:13:22.842183 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:22.842184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:22.842213 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:22.842277 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:22.842919 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:22.842922 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:22.842924 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:22.842933 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:22.842958 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:22.842967 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:22.843031 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:22.843127 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 05:13:22.843157 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:13:22.843348 30011 raft_consensus.cc:374] T 00000000000000000000000000000000 P 864699ced19e4dcba8bac1aa4a943181 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:22.843411 30011 raft_consensus.cc:729] T 00000000000000000000000000000000 P 864699ced19e4dcba8bac1aa4a943181 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 864699ced19e4dcba8bac1aa4a943181, State: Initialized, Role: FOLLOWER
I20260807 05:13:22.843866 30011 consensus_queue.cc:260] T 00000000000000000000000000000000 P 864699ced19e4dcba8bac1aa4a943181 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864699ced19e4dcba8bac1aa4a943181" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46585 } }
I20260807 05:13:22.844028 30011 raft_consensus.cc:388] T 00000000000000000000000000000000 P 864699ced19e4dcba8bac1aa4a943181 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:22.844115 30011 raft_consensus.cc:482] T 00000000000000000000000000000000 P 864699ced19e4dcba8bac1aa4a943181 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:22.844215 30011 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 864699ced19e4dcba8bac1aa4a943181 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:22.845109 30011 raft_consensus.cc:504] T 00000000000000000000000000000000 P 864699ced19e4dcba8bac1aa4a943181 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864699ced19e4dcba8bac1aa4a943181" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46585 } }
I20260807 05:13:22.845422 30011 leader_election.cc:302] T 00000000000000000000000000000000 P 864699ced19e4dcba8bac1aa4a943181 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 864699ced19e4dcba8bac1aa4a943181; no voters:
I20260807 05:13:22.845768 30011 leader_election.cc:288] T 00000000000000000000000000000000 P 864699ced19e4dcba8bac1aa4a943181 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:22.845854 30022 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 864699ced19e4dcba8bac1aa4a943181 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:22.846127 30022 raft_consensus.cc:686] T 00000000000000000000000000000000 P 864699ced19e4dcba8bac1aa4a943181 [term 1 LEADER]: Becoming Leader. State: Replica: 864699ced19e4dcba8bac1aa4a943181, State: Running, Role: LEADER
I20260807 05:13:22.846463 30022 consensus_queue.cc:237] T 00000000000000000000000000000000 P 864699ced19e4dcba8bac1aa4a943181 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864699ced19e4dcba8bac1aa4a943181" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46585 } }
I20260807 05:13:22.847007 30011 sys_catalog.cc:564] T 00000000000000000000000000000000 P 864699ced19e4dcba8bac1aa4a943181 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:22.848086 30023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 864699ced19e4dcba8bac1aa4a943181 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "864699ced19e4dcba8bac1aa4a943181" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864699ced19e4dcba8bac1aa4a943181" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46585 } } }
I20260807 05:13:22.848127 30024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 864699ced19e4dcba8bac1aa4a943181 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 864699ced19e4dcba8bac1aa4a943181. Latest consensus state: current_term: 1 leader_uuid: "864699ced19e4dcba8bac1aa4a943181" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864699ced19e4dcba8bac1aa4a943181" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46585 } } }
I20260807 05:13:22.848290 30024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 864699ced19e4dcba8bac1aa4a943181 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:22.848286 30023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 864699ced19e4dcba8bac1aa4a943181 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:22.849046 30032 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:22.851928 30032 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:22.856854 30032 catalog_manager.cc:1347] Generated new cluster ID: 8fbd1601a4c8491bad7fe83f5486bdb2
I20260807 05:13:22.856925 30032 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:22.881258 30032 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:22.881649 30032 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:22.896775 30032 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 864699ced19e4dcba8bac1aa4a943181: Generated new TSK 0
I20260807 05:13:22.897670 30032 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:13:22.905195 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 29938
2026-08-07T05:13:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1339 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:13:22 dist-test-slave-49h3 krb5kdc[30043](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:13:22 dist-test-slave-49h3 krb5kdc[30043](info): set up 2 sockets
Aug 07 05:13:22 dist-test-slave-49h3 krb5kdc[30043](info): commencing operation
krb5kdc: starting...
W20260807 05:13:23.881625 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.956s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:13:23 dist-test-slave-49h3 krb5kdc[30043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:13:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:13:24.001807 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41583
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41583
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:24.112773 30057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:24.113041 30057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:24.113086 30057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:24.117086 30057 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:24.117158 30057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:24.117180 30057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:24.117213 30057 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:24.117238 30057 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:24.122468 30057 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:24.123517 30057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:24.124526 30057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:24.132944 30066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:24.132957 30064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:24.132937 30063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:24.133165 30057 server_base.cc:1034] running on GCE node
I20260807 05:13:24.133822 30057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:24.134969 30057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:24.136137 30057 hybrid_clock.cc:648] HybridClock initialized: now 1786079604136127 us; error 31 us; skew 500 ppm
Aug 07 05:13:24 dist-test-slave-49h3 krb5kdc[30043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079604, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:24.138958 30057 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:13:24.139824 30057 webserver.cc:460] Webserver started at http://127.0.0.1:39315/ using document root <none> and password file <none>
I20260807 05:13:24.140339 30057 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:24.140398 30057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:24.140568 30057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:24.142590 30057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "f62a2cc0ce2d47919ce14c936b628df9"
format_stamp: "Formatted at 2026-08-07 05:13:24 on dist-test-slave-49h3"
server_key: "83038d4c32bd06f71d36470ef403cf17"
server_key_iv: "bb6fde95eff66d6cdf519701e00cf1ca"
server_key_version: "encryptionkey@0"
I20260807 05:13:24.143110 30057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "f62a2cc0ce2d47919ce14c936b628df9"
format_stamp: "Formatted at 2026-08-07 05:13:24 on dist-test-slave-49h3"
server_key: "83038d4c32bd06f71d36470ef403cf17"
server_key_iv: "bb6fde95eff66d6cdf519701e00cf1ca"
server_key_version: "encryptionkey@0"
I20260807 05:13:24.146715 30057 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:13:24.148996 30073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:24.150193 30057 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:13:24.150368 30057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "f62a2cc0ce2d47919ce14c936b628df9"
format_stamp: "Formatted at 2026-08-07 05:13:24 on dist-test-slave-49h3"
server_key: "83038d4c32bd06f71d36470ef403cf17"
server_key_iv: "bb6fde95eff66d6cdf519701e00cf1ca"
server_key_version: "encryptionkey@0"
I20260807 05:13:24.150493 30057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:24.167243 30057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:24.169991 30057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:24.170171 30057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:24.176640 30057 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41583
I20260807 05:13:24.176683 30129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41583 every 8 connection(s)
I20260807 05:13:24.177596 30057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:24.177953 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 30057
I20260807 05:13:24.178084 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:24.178354 8257 external_mini_cluster.cc:1428] Setting key a929a76618972cdd371c6d24de29e53d
I20260807 05:13:24.180483 30130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:24.186746 30130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f62a2cc0ce2d47919ce14c936b628df9: Bootstrap starting.
Aug 07 05:13:24 dist-test-slave-49h3 krb5kdc[30043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 05:13:24.188863 30130 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f62a2cc0ce2d47919ce14c936b628df9: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:24.189568 30130 log.cc:824] T 00000000000000000000000000000000 P f62a2cc0ce2d47919ce14c936b628df9: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:24.191761 30130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f62a2cc0ce2d47919ce14c936b628df9: No bootstrap required, opened a new log
I20260807 05:13:24.192313 30133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:24.179658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43907 (local address 127.0.0.1:41583)
0807 05:13:24.180050 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:24.180064 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:24.180100 (+ 36us) server_negotiation.cc:407] Connection header received
0807 05:13:24.180751 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:24.180768 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:24.181083 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:24.181351 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:24.182259 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:24.183240 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:24.184393 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:24.184892 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:24.185011 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:24.187735 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:24.187740 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:24.187752 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:24.187778 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:24.189345 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:24.189782 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:24.189788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:24.189794 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:24.189938 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:24.190233 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:24.190235 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:24.190236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:24.190700 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:24.190888 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:24.191106 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 05:13:24.191398 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":118,"threads_started":1}
I20260807 05:13:24.194921 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:13:24.195390 30130 raft_consensus.cc:348] T 00000000000000000000000000000000 P f62a2cc0ce2d47919ce14c936b628df9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f62a2cc0ce2d47919ce14c936b628df9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41583 } }
I20260807 05:13:24.195669 30130 raft_consensus.cc:374] T 00000000000000000000000000000000 P f62a2cc0ce2d47919ce14c936b628df9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:24.195739 30130 raft_consensus.cc:729] T 00000000000000000000000000000000 P f62a2cc0ce2d47919ce14c936b628df9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f62a2cc0ce2d47919ce14c936b628df9, State: Initialized, Role: FOLLOWER
I20260807 05:13:24.196194 30130 consensus_queue.cc:260] T 00000000000000000000000000000000 P f62a2cc0ce2d47919ce14c936b628df9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f62a2cc0ce2d47919ce14c936b628df9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41583 } }
I20260807 05:13:24.196373 30130 raft_consensus.cc:388] T 00000000000000000000000000000000 P f62a2cc0ce2d47919ce14c936b628df9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:24.196450 30130 raft_consensus.cc:482] T 00000000000000000000000000000000 P f62a2cc0ce2d47919ce14c936b628df9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:24.196553 30130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f62a2cc0ce2d47919ce14c936b628df9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:24.197564 30130 raft_consensus.cc:504] T 00000000000000000000000000000000 P f62a2cc0ce2d47919ce14c936b628df9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f62a2cc0ce2d47919ce14c936b628df9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41583 } }
I20260807 05:13:24.197877 30130 leader_election.cc:302] T 00000000000000000000000000000000 P f62a2cc0ce2d47919ce14c936b628df9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f62a2cc0ce2d47919ce14c936b628df9; no voters:
I20260807 05:13:24.198143 30130 leader_election.cc:288] T 00000000000000000000000000000000 P f62a2cc0ce2d47919ce14c936b628df9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:24.198262 30140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f62a2cc0ce2d47919ce14c936b628df9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:24.198498 30140 raft_consensus.cc:686] T 00000000000000000000000000000000 P f62a2cc0ce2d47919ce14c936b628df9 [term 1 LEADER]: Becoming Leader. State: Replica: f62a2cc0ce2d47919ce14c936b628df9, State: Running, Role: LEADER
I20260807 05:13:24.198879 30140 consensus_queue.cc:237] T 00000000000000000000000000000000 P f62a2cc0ce2d47919ce14c936b628df9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f62a2cc0ce2d47919ce14c936b628df9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41583 } }
I20260807 05:13:24.199326 30130 sys_catalog.cc:564] T 00000000000000000000000000000000 P f62a2cc0ce2d47919ce14c936b628df9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:24.200060 30140 sys_catalog.cc:455] T 00000000000000000000000000000000 P f62a2cc0ce2d47919ce14c936b628df9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f62a2cc0ce2d47919ce14c936b628df9. Latest consensus state: current_term: 1 leader_uuid: "f62a2cc0ce2d47919ce14c936b628df9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f62a2cc0ce2d47919ce14c936b628df9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41583 } } }
I20260807 05:13:24.200239 30140 sys_catalog.cc:458] T 00000000000000000000000000000000 P f62a2cc0ce2d47919ce14c936b628df9 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:24.200497 30142 sys_catalog.cc:455] T 00000000000000000000000000000000 P f62a2cc0ce2d47919ce14c936b628df9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f62a2cc0ce2d47919ce14c936b628df9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f62a2cc0ce2d47919ce14c936b628df9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41583 } } }
I20260807 05:13:24.200596 30142 sys_catalog.cc:458] T 00000000000000000000000000000000 P f62a2cc0ce2d47919ce14c936b628df9 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:24.200652 30150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:24.204275 30150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:24.206028 30133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:24.197187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39353 (local address 127.0.0.1:41583)
0807 05:13:24.197308 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:24.197313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:24.197507 (+ 194us) server_negotiation.cc:407] Connection header received
0807 05:13:24.197631 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:24.197634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:24.197678 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:24.197740 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:24.198460 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:24.199189 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:24.200287 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:24.200611 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:24.200687 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:24.202394 (+ 1707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:24.202398 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:24.202401 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:24.202424 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:24.204098 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:24.204623 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:24.204627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:24.204629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:24.204736 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:24.205056 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:24.205060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:24.205062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:24.205286 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:24.205415 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:24.205740 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 05:13:24.205870 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":44}
I20260807 05:13:24.209982 30150 catalog_manager.cc:1347] Generated new cluster ID: 99f7ad097d2e4c329eef80fd8b7a76eb
I20260807 05:13:24.210047 30150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:24.225235 30150 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:24.226531 30150 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:24.238502 30150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f62a2cc0ce2d47919ce14c936b628df9: Generated new TSK 0
I20260807 05:13:24.239358 30150 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:13:24.249030 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 30057
2026-08-07T05:13:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1344 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:13:24 dist-test-slave-49h3 krb5kdc[30162](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:13:24 dist-test-slave-49h3 krb5kdc[30162](info): set up 2 sockets
Aug 07 05:13:24 dist-test-slave-49h3 krb5kdc[30162](info): commencing operation
krb5kdc: starting...
W20260807 05:13:25.229128 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.960s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:13:25 dist-test-slave-49h3 krb5kdc[30162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:13:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:13:25.350086 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33013
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:25.459918 30175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:25.460204 30175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:25.460247 30175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:25.464268 30175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:25.464347 30175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:25.464366 30175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:25.464381 30175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:25.464395 30175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:25.469844 30175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:25.470922 30175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:25.471948 30175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:25.480228 30182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:25.480216 30181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:25.480211 30184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:25.480600 30175 server_base.cc:1034] running on GCE node
I20260807 05:13:25.481156 30175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:25.482281 30175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:25.483424 30175 hybrid_clock.cc:648] HybridClock initialized: now 1786079605483404 us; error 35 us; skew 500 ppm
Aug 07 05:13:25 dist-test-slave-49h3 krb5kdc[30162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079605, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:25.490041 30175 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:13:25.491039 30175 webserver.cc:460] Webserver started at http://127.0.0.1:39293/ using document root <none> and password file <none>
I20260807 05:13:25.491593 30175 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:25.491659 30175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:25.491869 30175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:25.493981 30175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "f98f112393174078a9ebe2447a820182"
format_stamp: "Formatted at 2026-08-07 05:13:25 on dist-test-slave-49h3"
server_key: "0f6620b2099d0ffd7f011819546ef519"
server_key_iv: "f6c402d0e83c226cbed25fae13ed0f4b"
server_key_version: "encryptionkey@0"
I20260807 05:13:25.494520 30175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "f98f112393174078a9ebe2447a820182"
format_stamp: "Formatted at 2026-08-07 05:13:25 on dist-test-slave-49h3"
server_key: "0f6620b2099d0ffd7f011819546ef519"
server_key_iv: "f6c402d0e83c226cbed25fae13ed0f4b"
server_key_version: "encryptionkey@0"
I20260807 05:13:25.498191 30175 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:13:25.500505 30192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:25.501691 30175 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:13:25.501820 30175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "f98f112393174078a9ebe2447a820182"
format_stamp: "Formatted at 2026-08-07 05:13:25 on dist-test-slave-49h3"
server_key: "0f6620b2099d0ffd7f011819546ef519"
server_key_iv: "f6c402d0e83c226cbed25fae13ed0f4b"
server_key_version: "encryptionkey@0"
I20260807 05:13:25.501924 30175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:25.528172 30175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:25.531136 30175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:25.531322 30175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:25.538101 30175 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33013
I20260807 05:13:25.538184 30249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33013 every 8 connection(s)
I20260807 05:13:25.539024 30175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:25.541728 30250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:25.547127 30250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182: Bootstrap starting.
I20260807 05:13:25.547144 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 30175
I20260807 05:13:25.547272 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:25.547515 8257 external_mini_cluster.cc:1428] Setting key 254c0a9823b725d7552b32337e44df33
I20260807 05:13:25.549674 30250 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:25.550544 30250 log.cc:824] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:25.552757 30250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182: No bootstrap required, opened a new log
I20260807 05:13:25.557088 30253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:25.548768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53913 (local address 127.0.0.1:33013)
0807 05:13:25.549171 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:25.549189 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:25.549230 (+ 41us) server_negotiation.cc:407] Connection header received
0807 05:13:25.550083 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:25.550106 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:25.550521 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:25.550889 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:25.551230 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:25.552610 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:25.553536 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:25.554222 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:25.554384 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:25.555106 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:25.555113 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:25.555130 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:25.555666 (+ 536us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:25.555741 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:25.555773 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:25.555870 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:25.555938 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 05:13:25.556144 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":99,"threads_started":1}
I20260807 05:13:25.557055 30250 raft_consensus.cc:348] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f98f112393174078a9ebe2447a820182" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33013 } }
I20260807 05:13:25.557390 30250 raft_consensus.cc:374] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:25.557471 30250 raft_consensus.cc:729] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f98f112393174078a9ebe2447a820182, State: Initialized, Role: FOLLOWER
I20260807 05:13:25.557862 30250 consensus_queue.cc:260] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f98f112393174078a9ebe2447a820182" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33013 } }
I20260807 05:13:25.557991 30250 raft_consensus.cc:388] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:25.558054 30250 raft_consensus.cc:482] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:25.558158 30250 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:25.559127 30250 raft_consensus.cc:504] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f98f112393174078a9ebe2447a820182" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33013 } }
I20260807 05:13:25.559456 30250 leader_election.cc:302] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f98f112393174078a9ebe2447a820182; no voters:
I20260807 05:13:25.559748 30250 leader_election.cc:288] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:25.559732 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:13:25.559864 30255 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:25.560066 30255 raft_consensus.cc:686] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182 [term 1 LEADER]: Becoming Leader. State: Replica: f98f112393174078a9ebe2447a820182, State: Running, Role: LEADER
I20260807 05:13:25.560436 30255 consensus_queue.cc:237] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f98f112393174078a9ebe2447a820182" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33013 } }
I20260807 05:13:25.560922 30250 sys_catalog.cc:564] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:25.562031 30257 sys_catalog.cc:455] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f98f112393174078a9ebe2447a820182" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f98f112393174078a9ebe2447a820182" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33013 } } }
I20260807 05:13:25.562167 30257 sys_catalog.cc:458] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:25.562413 30257 sys_catalog.cc:455] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f98f112393174078a9ebe2447a820182. Latest consensus state: current_term: 1 leader_uuid: "f98f112393174078a9ebe2447a820182" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f98f112393174078a9ebe2447a820182" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33013 } } }
I20260807 05:13:25.562490 30257 sys_catalog.cc:458] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182 [sys.catalog]: This master's current role is: LEADER
W20260807 05:13:25.565902 30275 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:13:25.566104 30275 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:13:25.566249 30269 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:25.566481 30253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:25.562211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58307 (local address 127.0.0.1:33013)
0807 05:13:25.562314 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:25.562318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:25.562554 (+ 236us) server_negotiation.cc:407] Connection header received
0807 05:13:25.562737 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:25.562752 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:25.562796 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:25.562845 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:25.563246 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:25.563803 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:25.564771 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:25.565084 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:25.565200 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:25.566020 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:25.566024 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:25.566027 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:25.566039 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:25.566086 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:25.566101 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:25.566147 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:25.566307 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 05:13:25.566360 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 05:13:25.567173 30269 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:25.572357 30269 catalog_manager.cc:1347] Generated new cluster ID: ab29377e022348fcb838a125c94c5a87
I20260807 05:13:25.572432 30269 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:25.581589 30269 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:25.582796 30269 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:25.587175 30269 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f98f112393174078a9ebe2447a820182: Generated new TSK 0
I20260807 05:13:25.587940 30269 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:13:25.591831 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 30175
2026-08-07T05:13:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1342 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:13:25 dist-test-slave-49h3 krb5kdc[30281](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:13:25 dist-test-slave-49h3 krb5kdc[30281](info): set up 2 sockets
Aug 07 05:13:25 dist-test-slave-49h3 krb5kdc[30281](info): commencing operation
krb5kdc: starting...
W20260807 05:13:26.565478 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.954s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:13:26 dist-test-slave-49h3 krb5kdc[30281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:13:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:13:26.684466 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41569
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41569
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:26.792364 30297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:26.792634 30297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:26.792675 30297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:26.796573 30297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:26.796629 30297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:26.796646 30297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:26.796660 30297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:26.796679 30297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:26.801867 30297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:26.802866 30297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:26.803830 30297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:26.811889 30302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:26.811914 30306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:26.811909 30303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:26.812213 30297 server_base.cc:1034] running on GCE node
I20260807 05:13:26.812760 30297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:26.813853 30297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:26.814997 30297 hybrid_clock.cc:648] HybridClock initialized: now 1786079606814979 us; error 29 us; skew 500 ppm
Aug 07 05:13:26 dist-test-slave-49h3 krb5kdc[30281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079606, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:26.817556 30297 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:13:26.818446 30297 webserver.cc:460] Webserver started at http://127.0.0.1:45495/ using document root <none> and password file <none>
I20260807 05:13:26.818977 30297 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:26.819042 30297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:26.819238 30297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:26.821197 30297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "810d01fba03046eaa6366343efac5669"
format_stamp: "Formatted at 2026-08-07 05:13:26 on dist-test-slave-49h3"
server_key: "8bc1321225e5050ba294193b2c64eadf"
server_key_iv: "0a7ba92608c625e842eb0d7803518214"
server_key_version: "encryptionkey@0"
I20260807 05:13:26.821754 30297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "810d01fba03046eaa6366343efac5669"
format_stamp: "Formatted at 2026-08-07 05:13:26 on dist-test-slave-49h3"
server_key: "8bc1321225e5050ba294193b2c64eadf"
server_key_iv: "0a7ba92608c625e842eb0d7803518214"
server_key_version: "encryptionkey@0"
I20260807 05:13:26.825121 30297 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 05:13:26.827369 30314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:26.828441 30297 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 05:13:26.828549 30297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "810d01fba03046eaa6366343efac5669"
format_stamp: "Formatted at 2026-08-07 05:13:26 on dist-test-slave-49h3"
server_key: "8bc1321225e5050ba294193b2c64eadf"
server_key_iv: "0a7ba92608c625e842eb0d7803518214"
server_key_version: "encryptionkey@0"
I20260807 05:13:26.828639 30297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:26.832945 30297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:26.835494 30297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:26.835659 30297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:26.842165 30297 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41569
I20260807 05:13:26.842199 30368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41569 every 8 connection(s)
I20260807 05:13:26.843039 30297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:26.845692 30369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:26.850385 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 30297
I20260807 05:13:26.850525 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:26.850773 8257 external_mini_cluster.cc:1428] Setting key a1eb18380fcf2f2188be3311064ec0f5
I20260807 05:13:26.850947 30369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 810d01fba03046eaa6366343efac5669: Bootstrap starting.
I20260807 05:13:26.853278 30369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 810d01fba03046eaa6366343efac5669: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:26.854130 30369 log.cc:824] T 00000000000000000000000000000000 P 810d01fba03046eaa6366343efac5669: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:26.855590 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:26.851980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43689 (local address 127.0.0.1:41569)
0807 05:13:26.852387 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:26.852401 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:26.852436 (+ 35us) server_negotiation.cc:407] Connection header received
0807 05:13:26.853044 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:26.853059 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:26.853328 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:26.853657 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:26.854415 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:26.854420 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:26.854430 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:26.854441 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:26.854496 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:26.854515 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:26.854619 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:26.854735 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 05:13:26.854888 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":109,"threads_started":1}
I20260807 05:13:26.855960 30369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 810d01fba03046eaa6366343efac5669: No bootstrap required, opened a new log
I20260807 05:13:26.858230 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:13:26.860183 30369 raft_consensus.cc:348] T 00000000000000000000000000000000 P 810d01fba03046eaa6366343efac5669 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810d01fba03046eaa6366343efac5669" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41569 } }
I20260807 05:13:26.860425 30369 raft_consensus.cc:374] T 00000000000000000000000000000000 P 810d01fba03046eaa6366343efac5669 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:26.860481 30369 raft_consensus.cc:729] T 00000000000000000000000000000000 P 810d01fba03046eaa6366343efac5669 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 810d01fba03046eaa6366343efac5669, State: Initialized, Role: FOLLOWER
I20260807 05:13:26.860944 30369 consensus_queue.cc:260] T 00000000000000000000000000000000 P 810d01fba03046eaa6366343efac5669 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810d01fba03046eaa6366343efac5669" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41569 } }
I20260807 05:13:26.861109 30369 raft_consensus.cc:388] T 00000000000000000000000000000000 P 810d01fba03046eaa6366343efac5669 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:26.861183 30369 raft_consensus.cc:482] T 00000000000000000000000000000000 P 810d01fba03046eaa6366343efac5669 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:26.861281 30369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 810d01fba03046eaa6366343efac5669 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:26.862038 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:26.860437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53731 (local address 127.0.0.1:41569)
0807 05:13:26.860533 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:26.860537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:26.860788 (+ 251us) server_negotiation.cc:407] Connection header received
0807 05:13:26.860895 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:26.860897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:26.860925 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:26.860984 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:26.861706 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:26.861709 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:26.861710 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:26.861720 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:26.861745 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:26.861755 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:26.861819 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:26.861904 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 05:13:26.861932 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 05:13:26.862406 30369 raft_consensus.cc:504] T 00000000000000000000000000000000 P 810d01fba03046eaa6366343efac5669 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810d01fba03046eaa6366343efac5669" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41569 } }
I20260807 05:13:26.862761 30369 leader_election.cc:302] T 00000000000000000000000000000000 P 810d01fba03046eaa6366343efac5669 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 810d01fba03046eaa6366343efac5669; no voters:
I20260807 05:13:26.863059 30369 leader_election.cc:288] T 00000000000000000000000000000000 P 810d01fba03046eaa6366343efac5669 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:26.863178 30381 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 810d01fba03046eaa6366343efac5669 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:26.863385 30381 raft_consensus.cc:686] T 00000000000000000000000000000000 P 810d01fba03046eaa6366343efac5669 [term 1 LEADER]: Becoming Leader. State: Replica: 810d01fba03046eaa6366343efac5669, State: Running, Role: LEADER
I20260807 05:13:26.863713 30381 consensus_queue.cc:237] T 00000000000000000000000000000000 P 810d01fba03046eaa6366343efac5669 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810d01fba03046eaa6366343efac5669" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41569 } }
I20260807 05:13:26.864199 30369 sys_catalog.cc:564] T 00000000000000000000000000000000 P 810d01fba03046eaa6366343efac5669 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:26.865403 30383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 810d01fba03046eaa6366343efac5669 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 810d01fba03046eaa6366343efac5669. Latest consensus state: current_term: 1 leader_uuid: "810d01fba03046eaa6366343efac5669" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810d01fba03046eaa6366343efac5669" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41569 } } }
I20260807 05:13:26.865403 30382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 810d01fba03046eaa6366343efac5669 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "810d01fba03046eaa6366343efac5669" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810d01fba03046eaa6366343efac5669" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41569 } } }
I20260807 05:13:26.865566 30383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 810d01fba03046eaa6366343efac5669 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:26.865566 30382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 810d01fba03046eaa6366343efac5669 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:26.865960 30391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:26.869357 30391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:26.875041 30391 catalog_manager.cc:1347] Generated new cluster ID: c6e10ae31c2840e7b8603db2fef9a28c
I20260807 05:13:26.875102 30391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:26.906046 30391 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:26.906436 30391 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:26.916329 30391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 810d01fba03046eaa6366343efac5669: Generated new TSK 0
I20260807 05:13:26.917143 30391 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:13:26.933107 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 30297
2026-08-07T05:13:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1341 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:13:26 dist-test-slave-49h3 krb5kdc[30403](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:13:26 dist-test-slave-49h3 krb5kdc[30403](info): set up 2 sockets
Aug 07 05:13:26 dist-test-slave-49h3 krb5kdc[30403](info): commencing operation
krb5kdc: starting...
W20260807 05:13:27.917954 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.964s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:13:28 dist-test-slave-49h3 krb5kdc[30403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:13:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:13:28.041042 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33131
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:28.152064 30418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:28.152345 30418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:28.152406 30418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:28.156424 30418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:28.156507 30418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:28.156534 30418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:28.156575 30418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:28.156596 30418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:28.161921 30418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:28.162936 30418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:28.163894 30418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:28.172246 30424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:28.172240 30425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:28.172369 30418 server_base.cc:1034] running on GCE node
W20260807 05:13:28.172252 30427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:28.173055 30418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:28.174170 30418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:28.175340 30418 hybrid_clock.cc:648] HybridClock initialized: now 1786079608175316 us; error 34 us; skew 500 ppm
Aug 07 05:13:28 dist-test-slave-49h3 krb5kdc[30403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079608, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:28.177997 30418 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:13:28.178898 30418 webserver.cc:460] Webserver started at http://127.0.0.1:43235/ using document root <none> and password file <none>
I20260807 05:13:28.179437 30418 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:28.179507 30418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:28.179709 30418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:28.181783 30418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "5c5a75f87db3449bba75ff75f3e26538"
format_stamp: "Formatted at 2026-08-07 05:13:28 on dist-test-slave-49h3"
server_key: "8f5e6f3e2589414dbd99287b504a8c47"
server_key_iv: "c06fda7ca412858143c7afdb625f4041"
server_key_version: "encryptionkey@0"
I20260807 05:13:28.182329 30418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "5c5a75f87db3449bba75ff75f3e26538"
format_stamp: "Formatted at 2026-08-07 05:13:28 on dist-test-slave-49h3"
server_key: "8f5e6f3e2589414dbd99287b504a8c47"
server_key_iv: "c06fda7ca412858143c7afdb625f4041"
server_key_version: "encryptionkey@0"
I20260807 05:13:28.186167 30418 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 05:13:28.188582 30436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:28.189788 30418 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:13:28.189908 30418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "5c5a75f87db3449bba75ff75f3e26538"
format_stamp: "Formatted at 2026-08-07 05:13:28 on dist-test-slave-49h3"
server_key: "8f5e6f3e2589414dbd99287b504a8c47"
server_key_iv: "c06fda7ca412858143c7afdb625f4041"
server_key_version: "encryptionkey@0"
I20260807 05:13:28.190007 30418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:28.205391 30418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:28.208200 30418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:28.208384 30418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:28.215090 30418 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33131
I20260807 05:13:28.215147 30493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33131 every 8 connection(s)
I20260807 05:13:28.216033 30418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:28.217715 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 30418
I20260807 05:13:28.217850 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:28.218123 8257 external_mini_cluster.cc:1428] Setting key a57445140fa36b6797b302517a60a66d
I20260807 05:13:28.218947 30494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:28.225693 30494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c5a75f87db3449bba75ff75f3e26538: Bootstrap starting.
Aug 07 05:13:28 dist-test-slave-49h3 krb5kdc[30403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 05:13:28.228122 30494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c5a75f87db3449bba75ff75f3e26538: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:28.229005 30494 log.cc:824] T 00000000000000000000000000000000 P 5c5a75f87db3449bba75ff75f3e26538: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:28.231206 30494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c5a75f87db3449bba75ff75f3e26538: No bootstrap required, opened a new log
I20260807 05:13:28.232844 30497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:28.219597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:55635 (local address 127.0.0.1:33131)
0807 05:13:28.220024 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:28.220042 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:28.220084 (+ 42us) server_negotiation.cc:407] Connection header received
0807 05:13:28.220901 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:28.220924 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:28.221361 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:28.221765 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:28.222410 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:28.223871 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:28.224671 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:28.225345 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:28.225536 (+ 191us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:28.228067 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:28.228074 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:28.228091 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:28.228125 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:28.229855 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:28.230337 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:28.230343 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:28.230348 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:28.230487 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:28.230757 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:28.230760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:28.230762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:28.231255 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:28.231456 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:28.231693 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 05:13:28.231931 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":129,"threads_started":1}
I20260807 05:13:28.235481 30494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c5a75f87db3449bba75ff75f3e26538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c5a75f87db3449bba75ff75f3e26538" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33131 } }
I20260807 05:13:28.235754 30494 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c5a75f87db3449bba75ff75f3e26538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:28.235819 30494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c5a75f87db3449bba75ff75f3e26538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c5a75f87db3449bba75ff75f3e26538, State: Initialized, Role: FOLLOWER
I20260807 05:13:28.235894 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:13:28.236222 30494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c5a75f87db3449bba75ff75f3e26538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c5a75f87db3449bba75ff75f3e26538" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33131 } }
I20260807 05:13:28.236351 30494 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c5a75f87db3449bba75ff75f3e26538 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:28.236397 30494 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c5a75f87db3449bba75ff75f3e26538 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:28.236465 30494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c5a75f87db3449bba75ff75f3e26538 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:28.237367 30494 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c5a75f87db3449bba75ff75f3e26538 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c5a75f87db3449bba75ff75f3e26538" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33131 } }
I20260807 05:13:28.237730 30494 leader_election.cc:302] T 00000000000000000000000000000000 P 5c5a75f87db3449bba75ff75f3e26538 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c5a75f87db3449bba75ff75f3e26538; no voters:
I20260807 05:13:28.238090 30494 leader_election.cc:288] T 00000000000000000000000000000000 P 5c5a75f87db3449bba75ff75f3e26538 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:28.238171 30505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c5a75f87db3449bba75ff75f3e26538 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:28.238346 30505 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c5a75f87db3449bba75ff75f3e26538 [term 1 LEADER]: Becoming Leader. State: Replica: 5c5a75f87db3449bba75ff75f3e26538, State: Running, Role: LEADER
I20260807 05:13:28.238690 30505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c5a75f87db3449bba75ff75f3e26538 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c5a75f87db3449bba75ff75f3e26538" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33131 } }
I20260807 05:13:28.239372 30494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c5a75f87db3449bba75ff75f3e26538 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:28.240468 30506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c5a75f87db3449bba75ff75f3e26538 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c5a75f87db3449bba75ff75f3e26538" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c5a75f87db3449bba75ff75f3e26538" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33131 } } }
I20260807 05:13:28.240499 30507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c5a75f87db3449bba75ff75f3e26538 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c5a75f87db3449bba75ff75f3e26538. Latest consensus state: current_term: 1 leader_uuid: "5c5a75f87db3449bba75ff75f3e26538" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c5a75f87db3449bba75ff75f3e26538" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33131 } } }
I20260807 05:13:28.240634 30506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c5a75f87db3449bba75ff75f3e26538 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:28.240641 30507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c5a75f87db3449bba75ff75f3e26538 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:28.241102 30515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:28.244246 30515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:28.246779 30497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:28.237838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:47763 (local address 127.0.0.1:33131)
0807 05:13:28.237951 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:28.237955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:28.238112 (+ 157us) server_negotiation.cc:407] Connection header received
0807 05:13:28.238253 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:28.238256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:28.238302 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:28.238350 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:28.239171 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:28.239755 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:28.240902 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:28.241240 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:28.241334 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:28.243457 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:28.243461 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:28.243464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:28.243488 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:28.244808 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:28.245267 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:28.245270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:28.245271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:28.245343 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:28.245681 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:28.245684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:28.245685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:28.245852 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:28.245974 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:28.246529 (+ 555us) server_negotiation.cc:299] Negotiation successful
0807 05:13:28.246622 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 05:13:28.250001 30515 catalog_manager.cc:1347] Generated new cluster ID: 13d40235e7204a3e965830758dfdeaa6
I20260807 05:13:28.250073 30515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:28.288970 30515 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:28.290062 30515 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:28.297479 30515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c5a75f87db3449bba75ff75f3e26538: Generated new TSK 0
I20260807 05:13:28.298157 30515 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:13:28.315280 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 30418
W20260807 05:13:28.321863 30401 connection.cc:535] client connection to 127.0.0.1:33131 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33131): Connection reset by peer (error 104)
2026-08-07T05:13:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1382 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:13:28 dist-test-slave-49h3 krb5kdc[30527](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:13:28 dist-test-slave-49h3 krb5kdc[30527](info): set up 2 sockets
Aug 07 05:13:28 dist-test-slave-49h3 krb5kdc[30527](info): commencing operation
krb5kdc: starting...
W20260807 05:13:29.289530 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.954s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:13:29 dist-test-slave-49h3 krb5kdc[30527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:13:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:13:29.408478 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41313
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:29.518121 30540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:29.518404 30540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:29.518465 30540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:29.522467 30540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:29.522535 30540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:29.522562 30540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:29.522584 30540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:29.522611 30540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:29.527983 30540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:29.529040 30540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:29.530092 30540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:29.538245 30547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:29.538295 30540 server_base.cc:1034] running on GCE node
W20260807 05:13:29.538249 30548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:29.538245 30550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:29.539039 30540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:29.540093 30540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:29.541249 30540 hybrid_clock.cc:648] HybridClock initialized: now 1786079609541239 us; error 32 us; skew 500 ppm
Aug 07 05:13:29 dist-test-slave-49h3 krb5kdc[30527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079609, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:29.543938 30540 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:13:29.544766 30540 webserver.cc:460] Webserver started at http://127.0.0.1:32865/ using document root <none> and password file <none>
I20260807 05:13:29.545262 30540 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:29.545328 30540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:29.545545 30540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:29.547544 30540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "63882f267c8c4eb4a85212c45f384806"
format_stamp: "Formatted at 2026-08-07 05:13:29 on dist-test-slave-49h3"
server_key: "283dce7eb96a9de9150d972e058afae3"
server_key_iv: "982471fc81129bd4860b53f5983dfb5d"
server_key_version: "encryptionkey@0"
I20260807 05:13:29.548048 30540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "63882f267c8c4eb4a85212c45f384806"
format_stamp: "Formatted at 2026-08-07 05:13:29 on dist-test-slave-49h3"
server_key: "283dce7eb96a9de9150d972e058afae3"
server_key_iv: "982471fc81129bd4860b53f5983dfb5d"
server_key_version: "encryptionkey@0"
I20260807 05:13:29.551592 30540 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:13:29.554020 30557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:29.555227 30540 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:13:29.555344 30540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "63882f267c8c4eb4a85212c45f384806"
format_stamp: "Formatted at 2026-08-07 05:13:29 on dist-test-slave-49h3"
server_key: "283dce7eb96a9de9150d972e058afae3"
server_key_iv: "982471fc81129bd4860b53f5983dfb5d"
server_key_version: "encryptionkey@0"
I20260807 05:13:29.555449 30540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:29.581928 30540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:29.585265 30540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:29.585481 30540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:29.592341 30540 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41313
I20260807 05:13:29.592391 30612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41313 every 8 connection(s)
I20260807 05:13:29.593302 30540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:29.595000 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 30540
I20260807 05:13:29.595147 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:29.595470 8257 external_mini_cluster.cc:1428] Setting key 0217e4549340b7c33f27bd042fa0d0c9
I20260807 05:13:29.596312 30613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:29.602447 30613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63882f267c8c4eb4a85212c45f384806: Bootstrap starting.
I20260807 05:13:29.604046 30616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:29.596785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:44705 (local address 127.0.0.1:41313)
0807 05:13:29.597194 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:29.597209 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:29.597244 (+ 35us) server_negotiation.cc:407] Connection header received
0807 05:13:29.597888 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:29.597903 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:29.598212 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:29.598482 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:29.598833 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:29.599793 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:29.600921 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:29.601367 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:29.601510 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:29.602420 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:29.602425 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:29.602436 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:29.602840 (+ 404us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:29.602896 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:29.602920 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:29.603016 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:29.603196 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 05:13:29.603331 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":116,"threads_started":1}
I20260807 05:13:29.604686 30613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63882f267c8c4eb4a85212c45f384806: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:29.605420 30613 log.cc:824] T 00000000000000000000000000000000 P 63882f267c8c4eb4a85212c45f384806: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:29.606673 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:13:29.607306 30613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63882f267c8c4eb4a85212c45f384806: No bootstrap required, opened a new log
I20260807 05:13:29.611326 30613 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63882f267c8c4eb4a85212c45f384806 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63882f267c8c4eb4a85212c45f384806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41313 } }
I20260807 05:13:29.611603 30613 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63882f267c8c4eb4a85212c45f384806 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:29.611666 30613 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63882f267c8c4eb4a85212c45f384806 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63882f267c8c4eb4a85212c45f384806, State: Initialized, Role: FOLLOWER
I20260807 05:13:29.612102 30613 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63882f267c8c4eb4a85212c45f384806 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63882f267c8c4eb4a85212c45f384806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41313 } }
I20260807 05:13:29.612262 30613 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63882f267c8c4eb4a85212c45f384806 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:29.612377 30613 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63882f267c8c4eb4a85212c45f384806 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:29.612507 30616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:29.608552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:52215 (local address 127.0.0.1:41313)
0807 05:13:29.608649 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:29.608653 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:29.608889 (+ 236us) server_negotiation.cc:407] Connection header received
0807 05:13:29.608997 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:29.608999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:29.609036 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:29.609094 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:29.609378 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:29.609908 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:29.610983 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:29.611190 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:29.611254 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:29.612086 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:29.612090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:29.612093 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:29.612105 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:29.612133 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:29.612146 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:29.612238 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:29.612352 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 05:13:29.612389 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 05:13:29.612526 30613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63882f267c8c4eb4a85212c45f384806 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:29.613376 30613 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63882f267c8c4eb4a85212c45f384806 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63882f267c8c4eb4a85212c45f384806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41313 } }
I20260807 05:13:29.613708 30613 leader_election.cc:302] T 00000000000000000000000000000000 P 63882f267c8c4eb4a85212c45f384806 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63882f267c8c4eb4a85212c45f384806; no voters:
I20260807 05:13:29.613988 30613 leader_election.cc:288] T 00000000000000000000000000000000 P 63882f267c8c4eb4a85212c45f384806 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:29.614089 30623 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63882f267c8c4eb4a85212c45f384806 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:29.614326 30623 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63882f267c8c4eb4a85212c45f384806 [term 1 LEADER]: Becoming Leader. State: Replica: 63882f267c8c4eb4a85212c45f384806, State: Running, Role: LEADER
I20260807 05:13:29.614670 30623 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63882f267c8c4eb4a85212c45f384806 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63882f267c8c4eb4a85212c45f384806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41313 } }
I20260807 05:13:29.615084 30613 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63882f267c8c4eb4a85212c45f384806 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:29.616106 30625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63882f267c8c4eb4a85212c45f384806 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63882f267c8c4eb4a85212c45f384806. Latest consensus state: current_term: 1 leader_uuid: "63882f267c8c4eb4a85212c45f384806" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63882f267c8c4eb4a85212c45f384806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41313 } } }
I20260807 05:13:29.616225 30625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63882f267c8c4eb4a85212c45f384806 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:29.616591 30624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63882f267c8c4eb4a85212c45f384806 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63882f267c8c4eb4a85212c45f384806" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63882f267c8c4eb4a85212c45f384806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41313 } } }
I20260807 05:13:29.616683 30624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63882f267c8c4eb4a85212c45f384806 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:29.617144 30633 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:29.619774 30633 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:29.624477 30633 catalog_manager.cc:1347] Generated new cluster ID: 7a4d5b2ec25f47a9916d34ea1ec644ee
I20260807 05:13:29.624545 30633 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:29.653934 30633 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:29.654945 30633 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:29.666109 30633 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63882f267c8c4eb4a85212c45f384806: Generated new TSK 0
I20260807 05:13:29.666806 30633 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:13:29.671034 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 30540
W20260807 05:13:29.677248 30525 connection.cc:535] client connection to 127.0.0.1:41313 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41313): Connection reset by peer (error 104)
2026-08-07T05:13:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1355 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:13:29 dist-test-slave-49h3 krb5kdc[30647](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:13:29 dist-test-slave-49h3 krb5kdc[30647](info): set up 2 sockets
Aug 07 05:13:29 dist-test-slave-49h3 krb5kdc[30647](info): commencing operation
krb5kdc: starting...
W20260807 05:13:30.653255 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.962s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:13:30 dist-test-slave-49h3 krb5kdc[30647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:13:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:13:30.772027 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34137
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:30.880620 30661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:30.880903 30661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:30.880962 30661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:30.884941 30661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:30.885016 30661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:30.885044 30661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:30.885064 30661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:30.885084 30661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:30.890429 30661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:30.891520 30661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:30.892544 30661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:30.900877 30669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:30.900820 30667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:30.900996 30661 server_base.cc:1034] running on GCE node
W20260807 05:13:30.900892 30666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:30.901661 30661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:30.902760 30661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:30.903914 30661 hybrid_clock.cc:648] HybridClock initialized: now 1786079610903895 us; error 36 us; skew 500 ppm
Aug 07 05:13:30 dist-test-slave-49h3 krb5kdc[30647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079610, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:30.906548 30661 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:13:30.907469 30661 webserver.cc:460] Webserver started at http://127.0.0.1:46661/ using document root <none> and password file <none>
I20260807 05:13:30.907997 30661 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:30.908056 30661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:30.908226 30661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:30.910288 30661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "eee1c687c7fa4720af8811f735814ec7"
format_stamp: "Formatted at 2026-08-07 05:13:30 on dist-test-slave-49h3"
server_key: "51050824a9869715ea21ad53b522b796"
server_key_iv: "a1cd8385bef0b08b92df6ff818a157c9"
server_key_version: "encryptionkey@0"
I20260807 05:13:30.910797 30661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "eee1c687c7fa4720af8811f735814ec7"
format_stamp: "Formatted at 2026-08-07 05:13:30 on dist-test-slave-49h3"
server_key: "51050824a9869715ea21ad53b522b796"
server_key_iv: "a1cd8385bef0b08b92df6ff818a157c9"
server_key_version: "encryptionkey@0"
I20260807 05:13:30.914371 30661 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:13:30.916705 30676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:30.917949 30661 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:13:30.918063 30661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "eee1c687c7fa4720af8811f735814ec7"
format_stamp: "Formatted at 2026-08-07 05:13:30 on dist-test-slave-49h3"
server_key: "51050824a9869715ea21ad53b522b796"
server_key_iv: "a1cd8385bef0b08b92df6ff818a157c9"
server_key_version: "encryptionkey@0"
I20260807 05:13:30.918164 30661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:30.922644 30661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:30.925297 30661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:30.925511 30661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:30.932212 30661 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34137
I20260807 05:13:30.932225 30732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34137 every 8 connection(s)
I20260807 05:13:30.933121 30661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:30.935824 30733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:30.938118 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 30661
I20260807 05:13:30.938249 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:30.938500 8257 external_mini_cluster.cc:1428] Setting key 7b2f220e83acbd3fc00b87799f089dbc
I20260807 05:13:30.941382 30733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eee1c687c7fa4720af8811f735814ec7: Bootstrap starting.
I20260807 05:13:30.943297 30737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:30.939751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:56833 (local address 127.0.0.1:34137)
0807 05:13:30.940143 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:30.940159 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:30.940193 (+ 34us) server_negotiation.cc:407] Connection header received
0807 05:13:30.940806 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:30.940822 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:30.941083 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:30.941359 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:30.942161 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:30.942166 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:30.942176 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:30.942186 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:30.942236 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:30.942254 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:30.942366 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:30.942458 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 05:13:30.942593 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":104,"threads_started":1}
I20260807 05:13:30.943768 30733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eee1c687c7fa4720af8811f735814ec7: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:30.944535 30733 log.cc:824] T 00000000000000000000000000000000 P eee1c687c7fa4720af8811f735814ec7: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:30.946015 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:13:30.946887 30733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eee1c687c7fa4720af8811f735814ec7: No bootstrap required, opened a new log
I20260807 05:13:30.949796 30737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:30.948304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:43221 (local address 127.0.0.1:34137)
0807 05:13:30.948417 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:30.948422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:30.948611 (+ 189us) server_negotiation.cc:407] Connection header received
0807 05:13:30.948695 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:30.948698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:30.948733 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:30.948816 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:30.949418 (+ 602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:30.949421 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:30.949424 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:30.949435 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:30.949482 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:30.949496 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:30.949585 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:30.949636 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 05:13:30.949674 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 05:13:30.951148 30733 raft_consensus.cc:348] T 00000000000000000000000000000000 P eee1c687c7fa4720af8811f735814ec7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eee1c687c7fa4720af8811f735814ec7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34137 } }
I20260807 05:13:30.951370 30733 raft_consensus.cc:374] T 00000000000000000000000000000000 P eee1c687c7fa4720af8811f735814ec7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:30.951409 30733 raft_consensus.cc:729] T 00000000000000000000000000000000 P eee1c687c7fa4720af8811f735814ec7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eee1c687c7fa4720af8811f735814ec7, State: Initialized, Role: FOLLOWER
I20260807 05:13:30.951776 30733 consensus_queue.cc:260] T 00000000000000000000000000000000 P eee1c687c7fa4720af8811f735814ec7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eee1c687c7fa4720af8811f735814ec7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34137 } }
I20260807 05:13:30.951886 30733 raft_consensus.cc:388] T 00000000000000000000000000000000 P eee1c687c7fa4720af8811f735814ec7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:30.951931 30733 raft_consensus.cc:482] T 00000000000000000000000000000000 P eee1c687c7fa4720af8811f735814ec7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:30.952000 30733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eee1c687c7fa4720af8811f735814ec7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:30.952890 30733 raft_consensus.cc:504] T 00000000000000000000000000000000 P eee1c687c7fa4720af8811f735814ec7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eee1c687c7fa4720af8811f735814ec7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34137 } }
I20260807 05:13:30.953184 30733 leader_election.cc:302] T 00000000000000000000000000000000 P eee1c687c7fa4720af8811f735814ec7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eee1c687c7fa4720af8811f735814ec7; no voters:
I20260807 05:13:30.953437 30733 leader_election.cc:288] T 00000000000000000000000000000000 P eee1c687c7fa4720af8811f735814ec7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:30.953583 30744 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eee1c687c7fa4720af8811f735814ec7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:30.953783 30744 raft_consensus.cc:686] T 00000000000000000000000000000000 P eee1c687c7fa4720af8811f735814ec7 [term 1 LEADER]: Becoming Leader. State: Replica: eee1c687c7fa4720af8811f735814ec7, State: Running, Role: LEADER
I20260807 05:13:30.954145 30744 consensus_queue.cc:237] T 00000000000000000000000000000000 P eee1c687c7fa4720af8811f735814ec7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eee1c687c7fa4720af8811f735814ec7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34137 } }
I20260807 05:13:30.954627 30733 sys_catalog.cc:564] T 00000000000000000000000000000000 P eee1c687c7fa4720af8811f735814ec7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:30.955725 30745 sys_catalog.cc:455] T 00000000000000000000000000000000 P eee1c687c7fa4720af8811f735814ec7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eee1c687c7fa4720af8811f735814ec7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eee1c687c7fa4720af8811f735814ec7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34137 } } }
I20260807 05:13:30.955754 30746 sys_catalog.cc:455] T 00000000000000000000000000000000 P eee1c687c7fa4720af8811f735814ec7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eee1c687c7fa4720af8811f735814ec7. Latest consensus state: current_term: 1 leader_uuid: "eee1c687c7fa4720af8811f735814ec7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eee1c687c7fa4720af8811f735814ec7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34137 } } }
I20260807 05:13:30.955849 30745 sys_catalog.cc:458] T 00000000000000000000000000000000 P eee1c687c7fa4720af8811f735814ec7 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:30.955861 30746 sys_catalog.cc:458] T 00000000000000000000000000000000 P eee1c687c7fa4720af8811f735814ec7 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:30.956261 30754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:30.959347 30754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:30.965036 30754 catalog_manager.cc:1347] Generated new cluster ID: 805cf76bc2bf45058b8933d2af5d25ca
I20260807 05:13:30.965111 30754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:30.977257 30754 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:30.977617 30754 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:30.986752 30754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eee1c687c7fa4720af8811f735814ec7: Generated new TSK 0
I20260807 05:13:30.987452 30754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:13:30.998144 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 30661
2026-08-07T05:13:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:13:31 dist-test-slave-49h3 krb5kdc[30767](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:13:31 dist-test-slave-49h3 krb5kdc[30767](info): set up 2 sockets
Aug 07 05:13:31 dist-test-slave-49h3 krb5kdc[30767](info): commencing operation
krb5kdc: starting...
W20260807 05:13:31.985724 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.967s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:13:32 dist-test-slave-49h3 krb5kdc[30767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:13:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:13:32.110293 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40009
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:32.219394 30780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:32.219650 30780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:32.219709 30780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:32.223726 30780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:32.223793 30780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:32.223819 30780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:32.223851 30780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:32.223881 30780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:32.229144 30780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40009
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:32.230218 30780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:32.231208 30780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:32.239290 30785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:32.239343 30780 server_base.cc:1034] running on GCE node
W20260807 05:13:32.239364 30786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:32.239296 30788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:32.240025 30780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:32.241118 30780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:32.242308 30780 hybrid_clock.cc:648] HybridClock initialized: now 1786079612242283 us; error 36 us; skew 500 ppm
Aug 07 05:13:32 dist-test-slave-49h3 krb5kdc[30767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079612, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:32.245004 30780 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:13:32.245941 30780 webserver.cc:460] Webserver started at http://127.0.0.1:44319/ using document root <none> and password file <none>
I20260807 05:13:32.246511 30780 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:32.246593 30780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:32.246807 30780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:32.248843 30780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "5c2947803ab04617aff03dc69918937c"
format_stamp: "Formatted at 2026-08-07 05:13:32 on dist-test-slave-49h3"
server_key: "f8c25d5f417db2554fee975bea53d8d6"
server_key_iv: "289dc1fa42dc926e74af1ec6db981118"
server_key_version: "encryptionkey@0"
I20260807 05:13:32.249390 30780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "5c2947803ab04617aff03dc69918937c"
format_stamp: "Formatted at 2026-08-07 05:13:32 on dist-test-slave-49h3"
server_key: "f8c25d5f417db2554fee975bea53d8d6"
server_key_iv: "289dc1fa42dc926e74af1ec6db981118"
server_key_version: "encryptionkey@0"
I20260807 05:13:32.253028 30780 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:13:32.255438 30795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:32.256699 30780 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:13:32.256829 30780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "5c2947803ab04617aff03dc69918937c"
format_stamp: "Formatted at 2026-08-07 05:13:32 on dist-test-slave-49h3"
server_key: "f8c25d5f417db2554fee975bea53d8d6"
server_key_iv: "289dc1fa42dc926e74af1ec6db981118"
server_key_version: "encryptionkey@0"
I20260807 05:13:32.256932 30780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:32.267686 30780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:32.270784 30780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:32.270967 30780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:32.277531 30780 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40009
I20260807 05:13:32.277580 30850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40009 every 8 connection(s)
I20260807 05:13:32.278460 30780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:32.281050 30851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:32.286399 30851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c: Bootstrap starting.
I20260807 05:13:32.286984 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 30780
I20260807 05:13:32.287130 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:32.287413 8257 external_mini_cluster.cc:1428] Setting key d2e877756b57987f65c4bd71c079f2fc
I20260807 05:13:32.289111 30851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:32.290007 30851 log.cc:824] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:32.292155 30851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c: No bootstrap required, opened a new log
I20260807 05:13:32.296020 30851 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2947803ab04617aff03dc69918937c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40009 } }
I20260807 05:13:32.296245 30851 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:32.296312 30851 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c2947803ab04617aff03dc69918937c, State: Initialized, Role: FOLLOWER
Aug 07 05:13:32 dist-test-slave-49h3 krb5kdc[30767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 05:13:32.296730 30851 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2947803ab04617aff03dc69918937c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40009 } }
I20260807 05:13:32.296865 30851 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:32.296932 30851 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:32.297027 30851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:32.297952 30851 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2947803ab04617aff03dc69918937c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40009 } }
I20260807 05:13:32.298269 30851 leader_election.cc:302] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c2947803ab04617aff03dc69918937c; no voters:
I20260807 05:13:32.298556 30851 leader_election.cc:288] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:32.298642 30857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:32.298841 30857 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c [term 1 LEADER]: Becoming Leader. State: Replica: 5c2947803ab04617aff03dc69918937c, State: Running, Role: LEADER
I20260807 05:13:32.299185 30857 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2947803ab04617aff03dc69918937c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40009 } }
I20260807 05:13:32.299863 30851 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:32.300704 30858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c2947803ab04617aff03dc69918937c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2947803ab04617aff03dc69918937c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40009 } } }
I20260807 05:13:32.300845 30858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:32.301146 30859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c2947803ab04617aff03dc69918937c. Latest consensus state: current_term: 1 leader_uuid: "5c2947803ab04617aff03dc69918937c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2947803ab04617aff03dc69918937c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40009 } } }
I20260807 05:13:32.301230 30859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:32.303169 30855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:32.288735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:48497 (local address 127.0.0.1:40009)
0807 05:13:32.289165 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:32.289186 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:32.289229 (+ 43us) server_negotiation.cc:407] Connection header received
0807 05:13:32.290090 (+ 861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:32.290113 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:32.290525 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:32.290847 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:32.291466 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:32.292901 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:32.293803 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:32.294492 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:32.294668 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:32.297086 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:32.297106 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:32.297123 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:32.297154 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:32.298848 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:32.299462 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:32.299467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:32.299471 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:32.299576 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:32.299897 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:32.299900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:32.299901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:32.300391 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:32.301296 (+ 905us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:32.301985 (+ 689us) server_negotiation.cc:299] Negotiation successful
0807 05:13:32.302228 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":120,"threads_started":1}
W20260807 05:13:32.304711 30875 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:13:32.304778 30875 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:13:32.304867 30867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:32.305653 30867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:32.306459 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:13:32.310899 30867 catalog_manager.cc:1347] Generated new cluster ID: 1dc1372b19b34a8a86b309881f063873
I20260807 05:13:32.310958 30867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:32.318059 30855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:32.308693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:39345 (local address 127.0.0.1:40009)
0807 05:13:32.308795 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:32.308799 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:32.309047 (+ 248us) server_negotiation.cc:407] Connection header received
0807 05:13:32.309212 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:32.309215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:32.309257 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:32.309322 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:32.310139 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:32.310940 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:32.312095 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:32.312455 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:32.312529 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:32.314364 (+ 1835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:32.314368 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:32.314371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:32.314396 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:32.316028 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:32.316584 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:32.316588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:32.316590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:32.316684 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:32.317050 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:32.317053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:32.317055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:32.317252 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:32.317365 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:32.317807 (+ 442us) server_negotiation.cc:299] Negotiation successful
0807 05:13:32.317904 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 05:13:32.322666 30867 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:32.323827 30867 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:32.329098 30867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c2947803ab04617aff03dc69918937c: Generated new TSK 0
I20260807 05:13:32.329866 30867 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:13:32.333513 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 30780
W20260807 05:13:32.339465 30765 connection.cc:535] client connection to 127.0.0.1:40009 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40009): Connection reset by peer (error 104)
2026-08-07T05:13:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1334 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:13:32 dist-test-slave-49h3 krb5kdc[30885](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:13:32 dist-test-slave-49h3 krb5kdc[30885](info): set up 2 sockets
Aug 07 05:13:32 dist-test-slave-49h3 krb5kdc[30885](info): commencing operation
krb5kdc: starting...
W20260807 05:13:33.313369 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.960s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:13:33 dist-test-slave-49h3 krb5kdc[30885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:13:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:13:33.432256 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34715
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:33.543793 30899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:33.544078 30899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:33.544138 30899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:33.548154 30899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:33.548223 30899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:33.548250 30899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:33.548270 30899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:33.548291 30899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:33.553774 30899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34715
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:33.554836 30899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:33.555850 30899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:33.563760 30905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:33.563783 30904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:33.563783 30907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:33.563902 30899 server_base.cc:1034] running on GCE node
I20260807 05:13:33.564579 30899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:33.565675 30899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:33.566838 30899 hybrid_clock.cc:648] HybridClock initialized: now 1786079613566826 us; error 25 us; skew 500 ppm
Aug 07 05:13:33 dist-test-slave-49h3 krb5kdc[30885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079613, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:33.569469 30899 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:13:33.570329 30899 webserver.cc:460] Webserver started at http://127.0.0.1:44121/ using document root <none> and password file <none>
I20260807 05:13:33.570850 30899 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:33.570917 30899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:33.571101 30899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:33.573076 30899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "f9ffe41e99194c0b9576f67a0fc9547c"
format_stamp: "Formatted at 2026-08-07 05:13:33 on dist-test-slave-49h3"
server_key: "f9c12070aba83ae5a6ff5dc76371eaf6"
server_key_iv: "55980f8fb24436212f5ce9704f2bde6a"
server_key_version: "encryptionkey@0"
I20260807 05:13:33.573659 30899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "f9ffe41e99194c0b9576f67a0fc9547c"
format_stamp: "Formatted at 2026-08-07 05:13:33 on dist-test-slave-49h3"
server_key: "f9c12070aba83ae5a6ff5dc76371eaf6"
server_key_iv: "55980f8fb24436212f5ce9704f2bde6a"
server_key_version: "encryptionkey@0"
I20260807 05:13:33.577214 30899 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:13:33.579543 30914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:33.580708 30899 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:13:33.580825 30899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "f9ffe41e99194c0b9576f67a0fc9547c"
format_stamp: "Formatted at 2026-08-07 05:13:33 on dist-test-slave-49h3"
server_key: "f9c12070aba83ae5a6ff5dc76371eaf6"
server_key_iv: "55980f8fb24436212f5ce9704f2bde6a"
server_key_version: "encryptionkey@0"
I20260807 05:13:33.580920 30899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:33.593334 30899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:33.596019 30899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:33.596201 30899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:33.602808 30899 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34715
I20260807 05:13:33.602854 30971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34715 every 8 connection(s)
I20260807 05:13:33.603744 30899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:33.606540 30972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:33.608783 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 30899
I20260807 05:13:33.608942 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:33.609248 8257 external_mini_cluster.cc:1428] Setting key d3eb0a5a818210cf8cd577ed495bc0dc
I20260807 05:13:33.612787 30972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9ffe41e99194c0b9576f67a0fc9547c: Bootstrap starting.
I20260807 05:13:33.615295 30972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9ffe41e99194c0b9576f67a0fc9547c: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:33.616178 30972 log.cc:824] T 00000000000000000000000000000000 P f9ffe41e99194c0b9576f67a0fc9547c: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:33.618140 30975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:33.610653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:50803 (local address 127.0.0.1:34715)
0807 05:13:33.611075 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:33.611091 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:33.611126 (+ 35us) server_negotiation.cc:407] Connection header received
0807 05:13:33.611779 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:33.611796 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:33.612133 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:33.612413 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:33.612800 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:33.613837 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:33.614998 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:33.615482 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:33.615595 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:33.616484 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:33.616489 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:33.616501 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:33.616885 (+ 384us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:33.616940 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:33.616961 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:33.617058 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:33.617243 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 05:13:33.617374 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":126,"threads_started":1}
I20260807 05:13:33.618248 30972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9ffe41e99194c0b9576f67a0fc9547c: No bootstrap required, opened a new log
I20260807 05:13:33.620971 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:13:33.622526 30972 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9ffe41e99194c0b9576f67a0fc9547c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ffe41e99194c0b9576f67a0fc9547c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34715 } }
I20260807 05:13:33.622816 30972 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9ffe41e99194c0b9576f67a0fc9547c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:33.622881 30972 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9ffe41e99194c0b9576f67a0fc9547c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9ffe41e99194c0b9576f67a0fc9547c, State: Initialized, Role: FOLLOWER
I20260807 05:13:33.623329 30972 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9ffe41e99194c0b9576f67a0fc9547c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ffe41e99194c0b9576f67a0fc9547c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34715 } }
I20260807 05:13:33.623495 30972 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9ffe41e99194c0b9576f67a0fc9547c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:33.623569 30972 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9ffe41e99194c0b9576f67a0fc9547c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:33.623677 30972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9ffe41e99194c0b9576f67a0fc9547c [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:33.624709 30972 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9ffe41e99194c0b9576f67a0fc9547c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ffe41e99194c0b9576f67a0fc9547c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34715 } }
I20260807 05:13:33.625038 30972 leader_election.cc:302] T 00000000000000000000000000000000 P f9ffe41e99194c0b9576f67a0fc9547c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9ffe41e99194c0b9576f67a0fc9547c; no voters:
I20260807 05:13:33.625335 30972 leader_election.cc:288] T 00000000000000000000000000000000 P f9ffe41e99194c0b9576f67a0fc9547c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:33.625481 30982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9ffe41e99194c0b9576f67a0fc9547c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:33.625747 30982 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9ffe41e99194c0b9576f67a0fc9547c [term 1 LEADER]: Becoming Leader. State: Replica: f9ffe41e99194c0b9576f67a0fc9547c, State: Running, Role: LEADER
I20260807 05:13:33.626129 30982 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9ffe41e99194c0b9576f67a0fc9547c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ffe41e99194c0b9576f67a0fc9547c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34715 } }
I20260807 05:13:33.626700 30972 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9ffe41e99194c0b9576f67a0fc9547c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:33.627687 30984 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9ffe41e99194c0b9576f67a0fc9547c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9ffe41e99194c0b9576f67a0fc9547c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ffe41e99194c0b9576f67a0fc9547c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34715 } } }
I20260807 05:13:33.627811 30975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:33.623430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:37297 (local address 127.0.0.1:34715)
0807 05:13:33.623532 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:33.623536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:33.623551 (+ 15us) server_negotiation.cc:407] Connection header received
0807 05:13:33.623663 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:33.623666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:33.623710 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:33.623793 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:33.624091 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:33.624624 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:33.625598 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:33.625941 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:33.626042 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:33.627108 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:33.627111 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:33.627114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:33.627128 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:33.627157 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:33.627170 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:33.627245 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:33.627576 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 05:13:33.627613 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 05:13:33.627836 30984 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9ffe41e99194c0b9576f67a0fc9547c [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:33.628058 30984 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9ffe41e99194c0b9576f67a0fc9547c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9ffe41e99194c0b9576f67a0fc9547c. Latest consensus state: current_term: 1 leader_uuid: "f9ffe41e99194c0b9576f67a0fc9547c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ffe41e99194c0b9576f67a0fc9547c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34715 } } }
I20260807 05:13:33.628134 30984 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9ffe41e99194c0b9576f67a0fc9547c [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:33.628218 30992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:33.631268 30992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:33.637320 30992 catalog_manager.cc:1347] Generated new cluster ID: c9d1fd3b00ef4984b266e873b6b6085f
I20260807 05:13:33.637382 30992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:33.643169 30992 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:33.644016 30992 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:33.654855 30992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9ffe41e99194c0b9576f67a0fc9547c: Generated new TSK 0
I20260807 05:13:33.655540 30992 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:13:33.661827 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 30899
W20260807 05:13:33.668167 30883 connection.cc:535] client connection to 127.0.0.1:34715 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34715): Connection reset by peer (error 104)
2026-08-07T05:13:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1328 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:13:33 dist-test-slave-49h3 krb5kdc[31004](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:13:33 dist-test-slave-49h3 krb5kdc[31004](info): set up 2 sockets
Aug 07 05:13:33 dist-test-slave-49h3 krb5kdc[31004](info): commencing operation
krb5kdc: starting...
W20260807 05:13:34.637269 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.955s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:13:34 dist-test-slave-49h3 krb5kdc[31004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:13:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:13:34.760882 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37763
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37763
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:34.870648 31020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:34.870930 31020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:34.870990 31020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:34.875028 31020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:34.875100 31020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:34.875128 31020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:34.875149 31020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:34.875175 31020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:34.880451 31020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37763
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:34.881559 31020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:34.882584 31020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:34.890712 31025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:34.890873 31020 server_base.cc:1034] running on GCE node
W20260807 05:13:34.890782 31028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:34.890719 31026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:34.891544 31020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:34.892633 31020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:34.893815 31020 hybrid_clock.cc:648] HybridClock initialized: now 1786079614893797 us; error 28 us; skew 500 ppm
Aug 07 05:13:34 dist-test-slave-49h3 krb5kdc[31004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079614, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:34.896606 31020 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:13:34.897608 31020 webserver.cc:460] Webserver started at http://127.0.0.1:46505/ using document root <none> and password file <none>
I20260807 05:13:34.898151 31020 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:34.898216 31020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:34.898413 31020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:34.900430 31020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "30bcdadd14ab4c65975644df65158705"
format_stamp: "Formatted at 2026-08-07 05:13:34 on dist-test-slave-49h3"
server_key: "b824c07d28a0e83c023ebbcc32e6c8ac"
server_key_iv: "4e8b5190d3a6246f5bb659b3461dfcb5"
server_key_version: "encryptionkey@0"
I20260807 05:13:34.900979 31020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "30bcdadd14ab4c65975644df65158705"
format_stamp: "Formatted at 2026-08-07 05:13:34 on dist-test-slave-49h3"
server_key: "b824c07d28a0e83c023ebbcc32e6c8ac"
server_key_iv: "4e8b5190d3a6246f5bb659b3461dfcb5"
server_key_version: "encryptionkey@0"
I20260807 05:13:34.904598 31020 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:13:34.906942 31036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:34.908192 31020 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:13:34.908315 31020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "30bcdadd14ab4c65975644df65158705"
format_stamp: "Formatted at 2026-08-07 05:13:34 on dist-test-slave-49h3"
server_key: "b824c07d28a0e83c023ebbcc32e6c8ac"
server_key_iv: "4e8b5190d3a6246f5bb659b3461dfcb5"
server_key_version: "encryptionkey@0"
I20260807 05:13:34.908411 31020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:34.912909 31020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:34.915947 31020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:34.916132 31020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:34.922979 31020 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37763
I20260807 05:13:34.922988 31091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37763 every 8 connection(s)
I20260807 05:13:34.923902 31020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:34.926615 31093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:34.927768 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 31020
I20260807 05:13:34.927925 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:34.928217 8257 external_mini_cluster.cc:1428] Setting key 920eea57028ac216281491e618cce286
I20260807 05:13:34.932643 31093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30bcdadd14ab4c65975644df65158705: Bootstrap starting.
I20260807 05:13:34.933197 31096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:34.929512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:39417 (local address 127.0.0.1:37763)
0807 05:13:34.929896 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:34.929911 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:34.929946 (+ 35us) server_negotiation.cc:407] Connection header received
0807 05:13:34.930560 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:34.930588 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:34.930864 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:34.931137 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:34.931981 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:34.931986 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:34.931997 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:34.932009 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:34.932067 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:34.932088 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:34.932204 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:34.932299 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 05:13:34.932443 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":99,"threads_started":1}
I20260807 05:13:34.934947 31093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30bcdadd14ab4c65975644df65158705: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:34.935617 31093 log.cc:824] T 00000000000000000000000000000000 P 30bcdadd14ab4c65975644df65158705: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:34.936079 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:13:34.937862 31093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30bcdadd14ab4c65975644df65158705: No bootstrap required, opened a new log
I20260807 05:13:34.939910 31096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:34.938342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:46299 (local address 127.0.0.1:37763)
0807 05:13:34.938456 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:34.938460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:34.938707 (+ 247us) server_negotiation.cc:407] Connection header received
0807 05:13:34.938826 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:34.938828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:34.938861 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:34.938924 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:34.939566 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:34.939569 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:34.939571 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:34.939581 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:34.939604 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:34.939614 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:34.939679 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:34.939764 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 05:13:34.939796 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 05:13:34.941965 31093 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30bcdadd14ab4c65975644df65158705 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30bcdadd14ab4c65975644df65158705" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37763 } }
I20260807 05:13:34.942179 31093 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30bcdadd14ab4c65975644df65158705 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:34.942235 31093 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30bcdadd14ab4c65975644df65158705 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30bcdadd14ab4c65975644df65158705, State: Initialized, Role: FOLLOWER
I20260807 05:13:34.942615 31093 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30bcdadd14ab4c65975644df65158705 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30bcdadd14ab4c65975644df65158705" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37763 } }
I20260807 05:13:34.942737 31093 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30bcdadd14ab4c65975644df65158705 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:34.942799 31093 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30bcdadd14ab4c65975644df65158705 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:34.942893 31093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30bcdadd14ab4c65975644df65158705 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:34.943846 31093 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30bcdadd14ab4c65975644df65158705 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30bcdadd14ab4c65975644df65158705" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37763 } }
I20260807 05:13:34.944152 31093 leader_election.cc:302] T 00000000000000000000000000000000 P 30bcdadd14ab4c65975644df65158705 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30bcdadd14ab4c65975644df65158705; no voters:
I20260807 05:13:34.944423 31093 leader_election.cc:288] T 00000000000000000000000000000000 P 30bcdadd14ab4c65975644df65158705 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:34.944527 31103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30bcdadd14ab4c65975644df65158705 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:34.944748 31103 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30bcdadd14ab4c65975644df65158705 [term 1 LEADER]: Becoming Leader. State: Replica: 30bcdadd14ab4c65975644df65158705, State: Running, Role: LEADER
I20260807 05:13:34.945058 31103 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30bcdadd14ab4c65975644df65158705 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30bcdadd14ab4c65975644df65158705" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37763 } }
I20260807 05:13:34.945600 31093 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30bcdadd14ab4c65975644df65158705 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:34.946744 31105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30bcdadd14ab4c65975644df65158705 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30bcdadd14ab4c65975644df65158705. Latest consensus state: current_term: 1 leader_uuid: "30bcdadd14ab4c65975644df65158705" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30bcdadd14ab4c65975644df65158705" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37763 } } }
I20260807 05:13:34.946892 31105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30bcdadd14ab4c65975644df65158705 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:34.946741 31104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30bcdadd14ab4c65975644df65158705 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30bcdadd14ab4c65975644df65158705" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30bcdadd14ab4c65975644df65158705" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37763 } } }
I20260807 05:13:34.947167 31104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30bcdadd14ab4c65975644df65158705 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:34.947324 31113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:34.950227 31113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:34.956287 31113 catalog_manager.cc:1347] Generated new cluster ID: 667b565ff7d148929efe7e1d4243b880
I20260807 05:13:34.956355 31113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:34.964982 31113 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:34.965337 31113 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:34.976665 31113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30bcdadd14ab4c65975644df65158705: Generated new TSK 0
I20260807 05:13:34.977656 31113 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:13:34.986274 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 31020
2026-08-07T05:13:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1325 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:13:35 dist-test-slave-49h3 krb5kdc[31125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:13:35 dist-test-slave-49h3 krb5kdc[31125](info): set up 2 sockets
Aug 07 05:13:35 dist-test-slave-49h3 krb5kdc[31125](info): commencing operation
krb5kdc: starting...
W20260807 05:13:35.970361 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.963s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:13:36 dist-test-slave-49h3 krb5kdc[31125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:13:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:13:36.092651 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:36521
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:46193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:36521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:36.203234 31138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:36.203503 31138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:36.203544 31138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:36.207484 31138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:36.207545 31138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:36.207563 31138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:36.207578 31138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:36.207592 31138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:36.212807 31138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:46193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:36521
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:36521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:36.213894 31138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:36.214883 31138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:36.223071 31144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:36.223112 31143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:36.223182 31146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:36.223212 31138 server_base.cc:1034] running on GCE node
I20260807 05:13:36.223914 31138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:36.225009 31138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:36.226169 31138 hybrid_clock.cc:648] HybridClock initialized: now 1786079616226151 us; error 34 us; skew 500 ppm
Aug 07 05:13:36 dist-test-slave-49h3 krb5kdc[31125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079616, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:36.233830 31138 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:13:36.234809 31138 webserver.cc:460] Webserver started at http://127.8.16.126:40753/ using document root <none> and password file <none>
I20260807 05:13:36.235371 31138 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:36.235442 31138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:36.235649 31138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:36.237771 31138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "1ec9d9c3646749e492000c9e7e8cba33"
format_stamp: "Formatted at 2026-08-07 05:13:36 on dist-test-slave-49h3"
server_key: "03925cf18629c1622543b3601144d679"
server_key_iv: "0da653aaed5ada3e62be9b93427ae00c"
server_key_version: "encryptionkey@0"
I20260807 05:13:36.238335 31138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "1ec9d9c3646749e492000c9e7e8cba33"
format_stamp: "Formatted at 2026-08-07 05:13:36 on dist-test-slave-49h3"
server_key: "03925cf18629c1622543b3601144d679"
server_key_iv: "0da653aaed5ada3e62be9b93427ae00c"
server_key_version: "encryptionkey@0"
I20260807 05:13:36.242062 31138 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:13:36.244366 31153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:36.245700 31138 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:13:36.245815 31138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "1ec9d9c3646749e492000c9e7e8cba33"
format_stamp: "Formatted at 2026-08-07 05:13:36 on dist-test-slave-49h3"
server_key: "03925cf18629c1622543b3601144d679"
server_key_iv: "0da653aaed5ada3e62be9b93427ae00c"
server_key_version: "encryptionkey@0"
I20260807 05:13:36.245921 31138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:36.263702 31138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:36.267021 31138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:36.267230 31138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:36.274268 31138 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:36521
I20260807 05:13:36.274291 31209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:36521 every 8 connection(s)
I20260807 05:13:36.275362 31138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:36.278232 31210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:36.279528 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 31138
I20260807 05:13:36.279693 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:36.279953 8257 external_mini_cluster.cc:1428] Setting key 29b876dbac03eb480f69994a3b6efc53
I20260807 05:13:36.284940 31210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ec9d9c3646749e492000c9e7e8cba33: Bootstrap starting.
I20260807 05:13:36.287564 31210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ec9d9c3646749e492000c9e7e8cba33: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:36.288359 31210 log.cc:824] T 00000000000000000000000000000000 P 1ec9d9c3646749e492000c9e7e8cba33: Log is configured to *not* fsync() on all Append() calls
Aug 07 05:13:36 dist-test-slave-49h3 krb5kdc[31125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:13:36.290611 31210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ec9d9c3646749e492000c9e7e8cba33: No bootstrap required, opened a new log
I20260807 05:13:36.295210 31213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:36.281263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55145 (local address 127.8.16.126:36521)
0807 05:13:36.281733 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:36.281753 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:36.281796 (+ 43us) server_negotiation.cc:407] Connection header received
0807 05:13:36.282608 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:36.282635 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:36.283064 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:36.283437 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:36.284131 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:36.285573 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:36.286464 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:36.287149 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:36.287302 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:36.290070 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:36.290076 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:36.290101 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:36.290132 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:36.291988 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:36.292506 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:36.292512 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:36.292518 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:36.292667 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:36.292944 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:36.292947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:36.292949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:36.293533 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:36.293750 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:36.293958 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 05:13:36.294202 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":149,"threads_started":1}
I20260807 05:13:36.295158 31210 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ec9d9c3646749e492000c9e7e8cba33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec9d9c3646749e492000c9e7e8cba33" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36521 } }
I20260807 05:13:36.295462 31210 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ec9d9c3646749e492000c9e7e8cba33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:36.295527 31210 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ec9d9c3646749e492000c9e7e8cba33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ec9d9c3646749e492000c9e7e8cba33, State: Initialized, Role: FOLLOWER
I20260807 05:13:36.295981 31210 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ec9d9c3646749e492000c9e7e8cba33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec9d9c3646749e492000c9e7e8cba33" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36521 } }
I20260807 05:13:36.296110 31210 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ec9d9c3646749e492000c9e7e8cba33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:36.296170 31210 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ec9d9c3646749e492000c9e7e8cba33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:36.296272 31210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ec9d9c3646749e492000c9e7e8cba33 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:36.297240 31210 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ec9d9c3646749e492000c9e7e8cba33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec9d9c3646749e492000c9e7e8cba33" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36521 } }
I20260807 05:13:36.297616 31210 leader_election.cc:302] T 00000000000000000000000000000000 P 1ec9d9c3646749e492000c9e7e8cba33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ec9d9c3646749e492000c9e7e8cba33; no voters:
I20260807 05:13:36.297950 31210 leader_election.cc:288] T 00000000000000000000000000000000 P 1ec9d9c3646749e492000c9e7e8cba33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:36.298065 31215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ec9d9c3646749e492000c9e7e8cba33 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:36.298295 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:13:36.298337 31215 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ec9d9c3646749e492000c9e7e8cba33 [term 1 LEADER]: Becoming Leader. State: Replica: 1ec9d9c3646749e492000c9e7e8cba33, State: Running, Role: LEADER
I20260807 05:13:36.298624 31215 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ec9d9c3646749e492000c9e7e8cba33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec9d9c3646749e492000c9e7e8cba33" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36521 } }
I20260807 05:13:36.299240 31210 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ec9d9c3646749e492000c9e7e8cba33 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:36.299618 31215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ec9d9c3646749e492000c9e7e8cba33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ec9d9c3646749e492000c9e7e8cba33. Latest consensus state: current_term: 1 leader_uuid: "1ec9d9c3646749e492000c9e7e8cba33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec9d9c3646749e492000c9e7e8cba33" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36521 } } }
I20260807 05:13:36.299722 31215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ec9d9c3646749e492000c9e7e8cba33 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:36.300011 31216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ec9d9c3646749e492000c9e7e8cba33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ec9d9c3646749e492000c9e7e8cba33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec9d9c3646749e492000c9e7e8cba33" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36521 } } }
I20260807 05:13:36.300096 31216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ec9d9c3646749e492000c9e7e8cba33 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:36.300326 31223 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:36.303498 31223 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:36.309719 31223 catalog_manager.cc:1347] Generated new cluster ID: e1883d26d95a40d8a4920205cedb16b0
I20260807 05:13:36.309789 31223 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:36.311234 31213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:36.300616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49025 (local address 127.8.16.126:36521)
0807 05:13:36.301037 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:36.301042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:36.301369 (+ 327us) server_negotiation.cc:407] Connection header received
0807 05:13:36.301571 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:36.301575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:36.301632 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:36.301708 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:36.302494 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:36.303297 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:36.304915 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:36.305266 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:36.305344 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:36.307358 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:36.307362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:36.307365 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:36.307393 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:36.309148 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:36.309785 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:36.309789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:36.309791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:36.309885 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:36.310327 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:36.310331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:36.310333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:36.310561 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:36.310722 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:36.310990 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 05:13:36.311086 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346}
I20260807 05:13:36.343292 31223 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:36.344363 31223 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:36.351505 31223 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ec9d9c3646749e492000c9e7e8cba33: Generated new TSK 0
I20260807 05:13:36.352206 31223 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:13:36.356986 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 31138
W20260807 05:13:36.362838 31123 connection.cc:535] client connection to 127.8.16.126:36521 recv error: Network error: failed to read from TLS socket (remote: 127.8.16.126:36521): Connection reset by peer (error 104)
2026-08-07T05:13:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1369 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:13:36 dist-test-slave-49h3 krb5kdc[31242](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:13:36 dist-test-slave-49h3 krb5kdc[31242](info): set up 2 sockets
Aug 07 05:13:36 dist-test-slave-49h3 krb5kdc[31242](info): commencing operation
krb5kdc: starting...
W20260807 05:13:37.332902 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.956s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:13:37 dist-test-slave-49h3 krb5kdc[31242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:13:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:13:37.453948 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:44943
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:45399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:44943
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:37.563674 31255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:37.563961 31255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:37.564020 31255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:37.568043 31255 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:37.568107 31255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:37.568135 31255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:37.568156 31255 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:37.568185 31255 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:37.573426 31255 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:45399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:44943
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:44943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:37.574512 31255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:37.575508 31255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:37.583788 31264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:37.583796 31261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:37.583807 31262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:37.583884 31255 server_base.cc:1034] running on GCE node
I20260807 05:13:37.584540 31255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:37.585625 31255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:37.586786 31255 hybrid_clock.cc:648] HybridClock initialized: now 1786079617586773 us; error 26 us; skew 500 ppm
Aug 07 05:13:37 dist-test-slave-49h3 krb5kdc[31242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079617, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:37.589344 31255 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:13:37.590338 31255 webserver.cc:460] Webserver started at http://127.8.16.126:41405/ using document root <none> and password file <none>
I20260807 05:13:37.590907 31255 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:37.590979 31255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:37.591197 31255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:37.593227 31255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "e3c0f6fc0a214dcc9b36dd79cc42348a"
format_stamp: "Formatted at 2026-08-07 05:13:37 on dist-test-slave-49h3"
server_key: "f1a0c915c99ef73d9eb710fa00b9a51f"
server_key_iv: "96d73114b85d92304d05213acacf48ba"
server_key_version: "encryptionkey@0"
I20260807 05:13:37.593802 31255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "e3c0f6fc0a214dcc9b36dd79cc42348a"
format_stamp: "Formatted at 2026-08-07 05:13:37 on dist-test-slave-49h3"
server_key: "f1a0c915c99ef73d9eb710fa00b9a51f"
server_key_iv: "96d73114b85d92304d05213acacf48ba"
server_key_version: "encryptionkey@0"
I20260807 05:13:37.597347 31255 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:13:37.599637 31271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:37.600787 31255 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:13:37.600893 31255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "e3c0f6fc0a214dcc9b36dd79cc42348a"
format_stamp: "Formatted at 2026-08-07 05:13:37 on dist-test-slave-49h3"
server_key: "f1a0c915c99ef73d9eb710fa00b9a51f"
server_key_iv: "96d73114b85d92304d05213acacf48ba"
server_key_version: "encryptionkey@0"
I20260807 05:13:37.600989 31255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:37.612300 31255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:37.614943 31255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:37.615126 31255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:37.621716 31255 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:44943
I20260807 05:13:37.621759 31325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:44943 every 8 connection(s)
I20260807 05:13:37.622614 31255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:37.625196 31326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:37.630306 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 31255
I20260807 05:13:37.630431 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:37.630686 8257 external_mini_cluster.cc:1428] Setting key db8ae33fe3b4dd17b49d3ad02a938f35
I20260807 05:13:37.630770 31326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a: Bootstrap starting.
I20260807 05:13:37.633193 31326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:37.634073 31326 log.cc:824] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:37.636181 31326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a: No bootstrap required, opened a new log
I20260807 05:13:37.639529 31329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:37.631960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57759 (local address 127.8.16.126:44943)
0807 05:13:37.632366 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:37.632382 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:37.632415 (+ 33us) server_negotiation.cc:407] Connection header received
0807 05:13:37.633057 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:37.633083 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:37.633477 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:37.633767 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:37.634122 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:37.635264 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:37.636417 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:37.636882 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:37.636997 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:37.637915 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:37.637920 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:37.637932 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:37.638299 (+ 367us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:37.638352 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:37.638374 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:37.638470 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:37.638656 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 05:13:37.638784 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":114,"threads_started":1}
I20260807 05:13:37.640079 31326 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c0f6fc0a214dcc9b36dd79cc42348a" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44943 } }
I20260807 05:13:37.640295 31326 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:37.640352 31326 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c0f6fc0a214dcc9b36dd79cc42348a, State: Initialized, Role: FOLLOWER
I20260807 05:13:37.640836 31326 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c0f6fc0a214dcc9b36dd79cc42348a" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44943 } }
I20260807 05:13:37.640992 31326 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:37.641058 31326 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:37.641150 31326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:37.642210 31326 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c0f6fc0a214dcc9b36dd79cc42348a" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44943 } }
I20260807 05:13:37.642376 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:13:37.642544 31326 leader_election.cc:302] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3c0f6fc0a214dcc9b36dd79cc42348a; no voters:
I20260807 05:13:37.642848 31326 leader_election.cc:288] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:37.642957 31331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:37.643214 31331 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a [term 1 LEADER]: Becoming Leader. State: Replica: e3c0f6fc0a214dcc9b36dd79cc42348a, State: Running, Role: LEADER
I20260807 05:13:37.643522 31331 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c0f6fc0a214dcc9b36dd79cc42348a" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44943 } }
I20260807 05:13:37.644058 31326 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:37.644779 31333 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3c0f6fc0a214dcc9b36dd79cc42348a. Latest consensus state: current_term: 1 leader_uuid: "e3c0f6fc0a214dcc9b36dd79cc42348a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c0f6fc0a214dcc9b36dd79cc42348a" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44943 } } }
I20260807 05:13:37.644948 31333 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:37.645143 31332 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3c0f6fc0a214dcc9b36dd79cc42348a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c0f6fc0a214dcc9b36dd79cc42348a" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 44943 } } }
I20260807 05:13:37.645267 31332 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a [sys.catalog]: This master's current role is: LEADER
W20260807 05:13:37.649360 31353 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:13:37.649533 31353 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:13:37.649628 31346 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:37.649878 31329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:37.645513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54903 (local address 127.8.16.126:44943)
0807 05:13:37.645635 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:37.645640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:37.645654 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:13:37.645691 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:37.645694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:37.645739 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:37.645821 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:37.646115 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:37.647349 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:37.648147 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:37.648371 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:37.648463 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:37.649243 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:37.649246 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:37.649255 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:37.649267 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:37.649292 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:37.649303 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:37.649369 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:37.649693 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 05:13:37.649731 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 05:13:37.650810 31346 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:37.656423 31346 catalog_manager.cc:1347] Generated new cluster ID: fa691c25347f47b4bbdea6e9f95407b3
I20260807 05:13:37.656495 31346 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:37.672315 31346 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:37.673393 31346 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:37.682040 31346 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3c0f6fc0a214dcc9b36dd79cc42348a: Generated new TSK 0
I20260807 05:13:37.682749 31346 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:13:37.690124 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 31255
W20260807 05:13:37.696226 31240 connection.cc:535] client connection to 127.8.16.126:44943 recv error: Network error: failed to read from TLS socket (remote: 127.8.16.126:44943): Connection reset by peer (error 104)
2026-08-07T05:13:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1333 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:13:37 dist-test-slave-49h3 krb5kdc[31359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:13:37 dist-test-slave-49h3 krb5kdc[31359](info): set up 2 sockets
Aug 07 05:13:37 dist-test-slave-49h3 krb5kdc[31359](info): commencing operation
krb5kdc: starting...
W20260807 05:13:38.677891 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.968s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:13:38 dist-test-slave-49h3 krb5kdc[31359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:13:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:13:38.797636 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:45193
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:34135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:45193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:38.906749 31373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:38.907023 31373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:38.907083 31373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:38.911085 31373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:38.911151 31373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:38.911177 31373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:38.911201 31373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:38.911230 31373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:38.916433 31373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:34135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:45193
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:45193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:38.917519 31373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:38.918536 31373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:38.926736 31378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:38.926774 31379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:38.926800 31381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:38.926959 31373 server_base.cc:1034] running on GCE node
I20260807 05:13:38.927591 31373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:38.928711 31373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:38.929867 31373 hybrid_clock.cc:648] HybridClock initialized: now 1786079618929856 us; error 30 us; skew 500 ppm
Aug 07 05:13:38 dist-test-slave-49h3 krb5kdc[31359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079618, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:38.932628 31373 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:13:38.933578 31373 webserver.cc:460] Webserver started at http://127.8.16.126:40311/ using document root <none> and password file <none>
I20260807 05:13:38.934108 31373 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:38.934173 31373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:38.934386 31373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:38.936623 31373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "f43a4a367b204e8fb82a0783b5c31b13"
format_stamp: "Formatted at 2026-08-07 05:13:38 on dist-test-slave-49h3"
server_key: "6467e9548a2cc8f57fce81d507ee26c3"
server_key_iv: "764e003bfc699967e901f676be4e89eb"
server_key_version: "encryptionkey@0"
I20260807 05:13:38.937187 31373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "f43a4a367b204e8fb82a0783b5c31b13"
format_stamp: "Formatted at 2026-08-07 05:13:38 on dist-test-slave-49h3"
server_key: "6467e9548a2cc8f57fce81d507ee26c3"
server_key_iv: "764e003bfc699967e901f676be4e89eb"
server_key_version: "encryptionkey@0"
I20260807 05:13:38.940836 31373 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260807 05:13:38.943308 31388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:38.944568 31373 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 05:13:38.944689 31373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "f43a4a367b204e8fb82a0783b5c31b13"
format_stamp: "Formatted at 2026-08-07 05:13:38 on dist-test-slave-49h3"
server_key: "6467e9548a2cc8f57fce81d507ee26c3"
server_key_iv: "764e003bfc699967e901f676be4e89eb"
server_key_version: "encryptionkey@0"
I20260807 05:13:38.944785 31373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:38.949287 31373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:38.952339 31373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:38.952509 31373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:38.959403 31373 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:45193
I20260807 05:13:38.959434 31441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:45193 every 8 connection(s)
I20260807 05:13:38.960347 31373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:38.963048 31443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:38.963961 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 31373
I20260807 05:13:38.964126 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:38.964438 8257 external_mini_cluster.cc:1428] Setting key 4e4dc37ea006e2df55e4abff2dc40ce9
I20260807 05:13:38.969391 31443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f43a4a367b204e8fb82a0783b5c31b13: Bootstrap starting.
I20260807 05:13:38.969807 31446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:38.965946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58283 (local address 127.8.16.126:45193)
0807 05:13:38.966428 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:38.966443 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:38.966478 (+ 35us) server_negotiation.cc:407] Connection header received
0807 05:13:38.967102 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:38.967118 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:38.967411 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:38.967722 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:38.968559 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:38.968563 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:38.968575 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:38.968585 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:38.968642 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:38.968660 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:38.968791 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:38.968888 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 05:13:38.969041 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":143,"threads_started":1}
I20260807 05:13:38.971730 31443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f43a4a367b204e8fb82a0783b5c31b13: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:38.972462 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:13:38.972532 31443 log.cc:824] T 00000000000000000000000000000000 P f43a4a367b204e8fb82a0783b5c31b13: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:38.974557 31443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f43a4a367b204e8fb82a0783b5c31b13: No bootstrap required, opened a new log
I20260807 05:13:38.975972 31446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:38.974456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34795 (local address 127.8.16.126:45193)
0807 05:13:38.974573 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:38.974577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:38.974769 (+ 192us) server_negotiation.cc:407] Connection header received
0807 05:13:38.974884 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:38.974887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:38.974925 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:38.975008 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:38.975620 (+ 612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:38.975623 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:38.975626 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:38.975637 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:38.975664 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:38.975684 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:38.975768 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:38.975817 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 05:13:38.975853 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 05:13:38.978420 31443 raft_consensus.cc:348] T 00000000000000000000000000000000 P f43a4a367b204e8fb82a0783b5c31b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f43a4a367b204e8fb82a0783b5c31b13" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 45193 } }
I20260807 05:13:38.978641 31443 raft_consensus.cc:374] T 00000000000000000000000000000000 P f43a4a367b204e8fb82a0783b5c31b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:38.978678 31443 raft_consensus.cc:729] T 00000000000000000000000000000000 P f43a4a367b204e8fb82a0783b5c31b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f43a4a367b204e8fb82a0783b5c31b13, State: Initialized, Role: FOLLOWER
I20260807 05:13:38.979025 31443 consensus_queue.cc:260] T 00000000000000000000000000000000 P f43a4a367b204e8fb82a0783b5c31b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f43a4a367b204e8fb82a0783b5c31b13" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 45193 } }
I20260807 05:13:38.979130 31443 raft_consensus.cc:388] T 00000000000000000000000000000000 P f43a4a367b204e8fb82a0783b5c31b13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:38.979173 31443 raft_consensus.cc:482] T 00000000000000000000000000000000 P f43a4a367b204e8fb82a0783b5c31b13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:38.979251 31443 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f43a4a367b204e8fb82a0783b5c31b13 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:38.980086 31443 raft_consensus.cc:504] T 00000000000000000000000000000000 P f43a4a367b204e8fb82a0783b5c31b13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f43a4a367b204e8fb82a0783b5c31b13" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 45193 } }
I20260807 05:13:38.980384 31443 leader_election.cc:302] T 00000000000000000000000000000000 P f43a4a367b204e8fb82a0783b5c31b13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f43a4a367b204e8fb82a0783b5c31b13; no voters:
I20260807 05:13:38.980633 31443 leader_election.cc:288] T 00000000000000000000000000000000 P f43a4a367b204e8fb82a0783b5c31b13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:38.980738 31453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f43a4a367b204e8fb82a0783b5c31b13 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:38.980970 31453 raft_consensus.cc:686] T 00000000000000000000000000000000 P f43a4a367b204e8fb82a0783b5c31b13 [term 1 LEADER]: Becoming Leader. State: Replica: f43a4a367b204e8fb82a0783b5c31b13, State: Running, Role: LEADER
I20260807 05:13:38.981331 31453 consensus_queue.cc:237] T 00000000000000000000000000000000 P f43a4a367b204e8fb82a0783b5c31b13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f43a4a367b204e8fb82a0783b5c31b13" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 45193 } }
I20260807 05:13:38.981916 31443 sys_catalog.cc:564] T 00000000000000000000000000000000 P f43a4a367b204e8fb82a0783b5c31b13 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:38.983125 31454 sys_catalog.cc:455] T 00000000000000000000000000000000 P f43a4a367b204e8fb82a0783b5c31b13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f43a4a367b204e8fb82a0783b5c31b13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f43a4a367b204e8fb82a0783b5c31b13" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 45193 } } }
I20260807 05:13:38.983142 31455 sys_catalog.cc:455] T 00000000000000000000000000000000 P f43a4a367b204e8fb82a0783b5c31b13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f43a4a367b204e8fb82a0783b5c31b13. Latest consensus state: current_term: 1 leader_uuid: "f43a4a367b204e8fb82a0783b5c31b13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f43a4a367b204e8fb82a0783b5c31b13" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 45193 } } }
I20260807 05:13:38.983325 31454 sys_catalog.cc:458] T 00000000000000000000000000000000 P f43a4a367b204e8fb82a0783b5c31b13 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:38.983331 31455 sys_catalog.cc:458] T 00000000000000000000000000000000 P f43a4a367b204e8fb82a0783b5c31b13 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:38.984237 31465 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:38.987274 31465 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:38.992663 31465 catalog_manager.cc:1347] Generated new cluster ID: d87fec30987e45728a982432f7d843dd
I20260807 05:13:38.992754 31465 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:39.005954 31465 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:39.006320 31465 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:39.029116 31465 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f43a4a367b204e8fb82a0783b5c31b13: Generated new TSK 0
I20260807 05:13:39.030085 31465 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:13:39.034956 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 31373
2026-08-07T05:13:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1344 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:13:39 dist-test-slave-49h3 krb5kdc[31475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:13:39 dist-test-slave-49h3 krb5kdc[31475](info): set up 2 sockets
Aug 07 05:13:39 dist-test-slave-49h3 krb5kdc[31475](info): commencing operation
krb5kdc: starting...
W20260807 05:13:40.009179 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.954s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:13:40 dist-test-slave-49h3 krb5kdc[31475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:13:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:13:40.130692 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:42061
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:45475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:42061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:40.239725 31489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:40.240010 31489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:40.240053 31489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:40.244061 31489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:40.244125 31489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:40.244144 31489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:40.244161 31489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:40.244189 31489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:40.249425 31489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:45475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:42061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:42061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:40.250521 31489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:40.251541 31489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:40.259871 31496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:40.259922 31489 server_base.cc:1034] running on GCE node
W20260807 05:13:40.259855 31495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:40.259840 31498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:40.260726 31489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:40.261885 31489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:40.263041 31489 hybrid_clock.cc:648] HybridClock initialized: now 1786079620263013 us; error 45 us; skew 500 ppm
Aug 07 05:13:40 dist-test-slave-49h3 krb5kdc[31475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079620, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:40.265733 31489 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:13:40.266722 31489 webserver.cc:460] Webserver started at http://127.8.16.126:35643/ using document root <none> and password file <none>
I20260807 05:13:40.267270 31489 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:40.267341 31489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:40.267549 31489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:40.269659 31489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "a45fff8db63042e0b2167a4b2522e01d"
format_stamp: "Formatted at 2026-08-07 05:13:40 on dist-test-slave-49h3"
server_key: "5ef6a346958a97c8cadd851e74a830a6"
server_key_iv: "17f0dc02bf153a98c7bf3dd1a86df0ae"
server_key_version: "encryptionkey@0"
I20260807 05:13:40.270217 31489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "a45fff8db63042e0b2167a4b2522e01d"
format_stamp: "Formatted at 2026-08-07 05:13:40 on dist-test-slave-49h3"
server_key: "5ef6a346958a97c8cadd851e74a830a6"
server_key_iv: "17f0dc02bf153a98c7bf3dd1a86df0ae"
server_key_version: "encryptionkey@0"
I20260807 05:13:40.273916 31489 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:13:40.276253 31505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:40.277511 31489 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:13:40.277629 31489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "a45fff8db63042e0b2167a4b2522e01d"
format_stamp: "Formatted at 2026-08-07 05:13:40 on dist-test-slave-49h3"
server_key: "5ef6a346958a97c8cadd851e74a830a6"
server_key_iv: "17f0dc02bf153a98c7bf3dd1a86df0ae"
server_key_version: "encryptionkey@0"
I20260807 05:13:40.277727 31489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:40.298887 31489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:40.301980 31489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:40.302170 31489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:40.309258 31489 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:42061
I20260807 05:13:40.309275 31561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:42061 every 8 connection(s)
I20260807 05:13:40.310269 31489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:40.312965 31562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:40.317376 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 31489
I20260807 05:13:40.317534 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:40.317800 8257 external_mini_cluster.cc:1428] Setting key 74dc896cbfa0bde2e0f7af345e821a8c
I20260807 05:13:40.318578 31562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a45fff8db63042e0b2167a4b2522e01d: Bootstrap starting.
I20260807 05:13:40.320791 31562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a45fff8db63042e0b2167a4b2522e01d: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:40.321708 31562 log.cc:824] T 00000000000000000000000000000000 P a45fff8db63042e0b2167a4b2522e01d: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:40.323778 31562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a45fff8db63042e0b2167a4b2522e01d: No bootstrap required, opened a new log
Aug 07 05:13:40 dist-test-slave-49h3 krb5kdc[31475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:13:40.328080 31562 raft_consensus.cc:348] T 00000000000000000000000000000000 P a45fff8db63042e0b2167a4b2522e01d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45fff8db63042e0b2167a4b2522e01d" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42061 } }
I20260807 05:13:40.328316 31562 raft_consensus.cc:374] T 00000000000000000000000000000000 P a45fff8db63042e0b2167a4b2522e01d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:40.328363 31562 raft_consensus.cc:729] T 00000000000000000000000000000000 P a45fff8db63042e0b2167a4b2522e01d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45fff8db63042e0b2167a4b2522e01d, State: Initialized, Role: FOLLOWER
I20260807 05:13:40.328785 31562 consensus_queue.cc:260] T 00000000000000000000000000000000 P a45fff8db63042e0b2167a4b2522e01d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45fff8db63042e0b2167a4b2522e01d" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42061 } }
I20260807 05:13:40.328907 31562 raft_consensus.cc:388] T 00000000000000000000000000000000 P a45fff8db63042e0b2167a4b2522e01d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:40.328967 31562 raft_consensus.cc:482] T 00000000000000000000000000000000 P a45fff8db63042e0b2167a4b2522e01d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:40.329056 31562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a45fff8db63042e0b2167a4b2522e01d [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:40.330102 31562 raft_consensus.cc:504] T 00000000000000000000000000000000 P a45fff8db63042e0b2167a4b2522e01d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45fff8db63042e0b2167a4b2522e01d" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42061 } }
I20260807 05:13:40.330464 31562 leader_election.cc:302] T 00000000000000000000000000000000 P a45fff8db63042e0b2167a4b2522e01d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a45fff8db63042e0b2167a4b2522e01d; no voters:
I20260807 05:13:40.330842 31562 leader_election.cc:288] T 00000000000000000000000000000000 P a45fff8db63042e0b2167a4b2522e01d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:40.330931 31567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a45fff8db63042e0b2167a4b2522e01d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:40.331099 31567 raft_consensus.cc:686] T 00000000000000000000000000000000 P a45fff8db63042e0b2167a4b2522e01d [term 1 LEADER]: Becoming Leader. State: Replica: a45fff8db63042e0b2167a4b2522e01d, State: Running, Role: LEADER
I20260807 05:13:40.331452 31567 consensus_queue.cc:237] T 00000000000000000000000000000000 P a45fff8db63042e0b2167a4b2522e01d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45fff8db63042e0b2167a4b2522e01d" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42061 } }
I20260807 05:13:40.332183 31562 sys_catalog.cc:564] T 00000000000000000000000000000000 P a45fff8db63042e0b2167a4b2522e01d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:40.332551 31565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:40.319192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38337 (local address 127.8.16.126:42061)
0807 05:13:40.319604 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:40.319620 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:40.319654 (+ 34us) server_negotiation.cc:407] Connection header received
0807 05:13:40.320284 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:40.320300 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:40.320631 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:40.320944 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:40.321905 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:40.323042 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:40.324219 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:40.324690 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:40.324818 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:40.327707 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:40.327713 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:40.327728 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:40.327768 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:40.329619 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:40.330079 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:40.330083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:40.330088 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:40.330242 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:40.330512 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:40.330514 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:40.330516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:40.330975 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:40.331183 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:40.331474 (+ 291us) server_negotiation.cc:299] Negotiation successful
0807 05:13:40.331774 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":117,"threads_started":1}
I20260807 05:13:40.332852 31567 sys_catalog.cc:455] T 00000000000000000000000000000000 P a45fff8db63042e0b2167a4b2522e01d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a45fff8db63042e0b2167a4b2522e01d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45fff8db63042e0b2167a4b2522e01d" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42061 } } }
I20260807 05:13:40.332976 31567 sys_catalog.cc:458] T 00000000000000000000000000000000 P a45fff8db63042e0b2167a4b2522e01d [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:40.333182 31567 sys_catalog.cc:455] T 00000000000000000000000000000000 P a45fff8db63042e0b2167a4b2522e01d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a45fff8db63042e0b2167a4b2522e01d. Latest consensus state: current_term: 1 leader_uuid: "a45fff8db63042e0b2167a4b2522e01d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45fff8db63042e0b2167a4b2522e01d" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 42061 } } }
I20260807 05:13:40.333266 31567 sys_catalog.cc:458] T 00000000000000000000000000000000 P a45fff8db63042e0b2167a4b2522e01d [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:40.333870 31576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:40.335595 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:13:40.336835 31576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:40.342056 31576 catalog_manager.cc:1347] Generated new cluster ID: a11e87089f354607ae8d0b06cdf08c57
I20260807 05:13:40.342126 31576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:40.346199 31565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:40.337500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56835 (local address 127.8.16.126:42061)
0807 05:13:40.337614 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:40.337618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:40.337949 (+ 331us) server_negotiation.cc:407] Connection header received
0807 05:13:40.338031 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:40.338034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:40.338082 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:40.338159 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:40.339040 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:40.339815 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:40.341346 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:40.341717 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:40.341817 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:40.343257 (+ 1440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:40.343261 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:40.343263 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:40.343286 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:40.344575 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:40.344973 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:40.344976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:40.344977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:40.345052 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:40.345315 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:40.345318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:40.345320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:40.345557 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:40.345675 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:40.345908 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 05:13:40.346023 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 05:13:40.354861 31576 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:40.355944 31576 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:40.361276 31576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a45fff8db63042e0b2167a4b2522e01d: Generated new TSK 0
I20260807 05:13:40.362109 31576 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:13:40.369794 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 31489
2026-08-07T05:13:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1335 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:13:40 dist-test-slave-49h3 krb5kdc[31594](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:13:40 dist-test-slave-49h3 krb5kdc[31594](info): set up 2 sockets
Aug 07 05:13:40 dist-test-slave-49h3 krb5kdc[31594](info): commencing operation
krb5kdc: starting...
W20260807 05:13:41.357641 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.967s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:13:41 dist-test-slave-49h3 krb5kdc[31594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:13:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:13:41.478068 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:41253
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:41429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:41253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:41.587978 31609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:41.588269 31609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:41.588312 31609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:41.592430 31609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:41.592514 31609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:41.592543 31609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:41.592567 31609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:41.592599 31609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:41.598205 31609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:41429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:41253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:41253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:41.599309 31609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:41.600323 31609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:41.608520 31615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:41.608753 31609 server_base.cc:1034] running on GCE node
W20260807 05:13:41.608526 31617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:41.608526 31614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:41.609423 31609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:41.610554 31609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:41.611716 31609 hybrid_clock.cc:648] HybridClock initialized: now 1786079621611698 us; error 30 us; skew 500 ppm
Aug 07 05:13:41 dist-test-slave-49h3 krb5kdc[31594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079621, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:41.617081 31609 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:13:41.618041 31609 webserver.cc:460] Webserver started at http://127.8.16.126:36579/ using document root <none> and password file <none>
I20260807 05:13:41.618587 31609 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:41.618654 31609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:41.618878 31609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:41.620900 31609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "3805363d2e4a4d94b481b4f84d52d75f"
format_stamp: "Formatted at 2026-08-07 05:13:41 on dist-test-slave-49h3"
server_key: "5b50a59de6ca8be1b8bbe0a0220930db"
server_key_iv: "6839781260ffbb6cb3680044cb6a2b84"
server_key_version: "encryptionkey@0"
I20260807 05:13:41.621423 31609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "3805363d2e4a4d94b481b4f84d52d75f"
format_stamp: "Formatted at 2026-08-07 05:13:41 on dist-test-slave-49h3"
server_key: "5b50a59de6ca8be1b8bbe0a0220930db"
server_key_iv: "6839781260ffbb6cb3680044cb6a2b84"
server_key_version: "encryptionkey@0"
I20260807 05:13:41.625056 31609 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:13:41.627382 31624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:41.628520 31609 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 05:13:41.628647 31609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "3805363d2e4a4d94b481b4f84d52d75f"
format_stamp: "Formatted at 2026-08-07 05:13:41 on dist-test-slave-49h3"
server_key: "5b50a59de6ca8be1b8bbe0a0220930db"
server_key_iv: "6839781260ffbb6cb3680044cb6a2b84"
server_key_version: "encryptionkey@0"
I20260807 05:13:41.628739 31609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:41.647004 31609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:41.649863 31609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:41.650053 31609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:41.656873 31609 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:41253
I20260807 05:13:41.656898 31679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:41253 every 8 connection(s)
I20260807 05:13:41.657848 31609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:41.660522 31680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:41.664994 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 31609
I20260807 05:13:41.665119 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:41.665392 8257 external_mini_cluster.cc:1428] Setting key 717a8fb7cce0a1cb9291ca8a08231af1
I20260807 05:13:41.666329 31680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f: Bootstrap starting.
I20260807 05:13:41.669059 31680 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:41.670006 31680 log.cc:824] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:41.672025 31680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f: No bootstrap required, opened a new log
I20260807 05:13:41.675266 31683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:41.666788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52289 (local address 127.8.16.126:41253)
0807 05:13:41.667223 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:41.667246 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:41.667288 (+ 42us) server_negotiation.cc:407] Connection header received
0807 05:13:41.668138 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:41.668162 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:41.668604 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:41.668959 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:41.669424 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:41.670735 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:41.671619 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:41.672299 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:41.672461 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:41.673225 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:41.673231 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:41.673247 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:41.673766 (+ 519us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:41.673835 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:41.673866 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:41.673963 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:41.674052 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 05:13:41.674273 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":122,"threads_started":1}
I20260807 05:13:41.676373 31680 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3805363d2e4a4d94b481b4f84d52d75f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41253 } }
I20260807 05:13:41.676597 31680 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:41.676633 31680 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3805363d2e4a4d94b481b4f84d52d75f, State: Initialized, Role: FOLLOWER
I20260807 05:13:41.677054 31680 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3805363d2e4a4d94b481b4f84d52d75f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41253 } }
I20260807 05:13:41.677223 31680 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:41.677294 31680 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:41.677399 31680 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:41.678085 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:13:41.678521 31680 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3805363d2e4a4d94b481b4f84d52d75f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41253 } }
I20260807 05:13:41.678898 31680 leader_election.cc:302] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3805363d2e4a4d94b481b4f84d52d75f; no voters:
I20260807 05:13:41.679270 31680 leader_election.cc:288] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:41.679365 31685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:41.679638 31685 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f [term 1 LEADER]: Becoming Leader. State: Replica: 3805363d2e4a4d94b481b4f84d52d75f, State: Running, Role: LEADER
I20260807 05:13:41.679991 31685 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3805363d2e4a4d94b481b4f84d52d75f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41253 } }
I20260807 05:13:41.680565 31680 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:41.681573 31689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3805363d2e4a4d94b481b4f84d52d75f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3805363d2e4a4d94b481b4f84d52d75f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41253 } } }
I20260807 05:13:41.681707 31689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:41.681923 31689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3805363d2e4a4d94b481b4f84d52d75f. Latest consensus state: current_term: 1 leader_uuid: "3805363d2e4a4d94b481b4f84d52d75f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3805363d2e4a4d94b481b4f84d52d75f" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 41253 } } }
I20260807 05:13:41.681986 31689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:41.685027 31683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:41.680585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43295 (local address 127.8.16.126:41253)
0807 05:13:41.680689 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:41.680693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:41.680707 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:13:41.680887 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:41.680890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:41.680940 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:41.681027 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:41.681330 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:41.682088 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:41.683433 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:41.683646 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:41.683732 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:41.684577 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:41.684581 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:41.684583 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:41.684596 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:41.684625 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:41.684638 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:41.684727 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:41.684840 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 05:13:41.684895 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260807 05:13:41.685518 31705 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:13:41.685586 31705 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:13:41.685927 31699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:41.686853 31699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:41.692539 31699 catalog_manager.cc:1347] Generated new cluster ID: 98077325d7794afe9791aa7efe2d9d86
I20260807 05:13:41.692612 31699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:41.710700 31699 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:41.711709 31699 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:41.716486 31699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3805363d2e4a4d94b481b4f84d52d75f: Generated new TSK 0
I20260807 05:13:41.717155 31699 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:13:41.723258 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 31609
2026-08-07T05:13:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1352 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:13:41 dist-test-slave-49h3 krb5kdc[31711](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:13:41 dist-test-slave-49h3 krb5kdc[31711](info): set up 2 sockets
Aug 07 05:13:41 dist-test-slave-49h3 krb5kdc[31711](info): commencing operation
krb5kdc: starting...
W20260807 05:13:42.705598 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.962s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:13:42 dist-test-slave-49h3 krb5kdc[31711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:13:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:13:42.824329 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:39803
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:41287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:39803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:42.934357 31724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:42.934634 31724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:42.934695 31724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:42.938737 31724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:42.938800 31724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:42.938827 31724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:42.938848 31724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:42.938872 31724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:42.944161 31724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:41287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:39803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:39803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:42.945181 31724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:42.946235 31724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:42.954308 31730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:42.954308 31729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:42.954319 31732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:42.954483 31724 server_base.cc:1034] running on GCE node
I20260807 05:13:42.955122 31724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:42.956185 31724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:42.957345 31724 hybrid_clock.cc:648] HybridClock initialized: now 1786079622957329 us; error 28 us; skew 500 ppm
Aug 07 05:13:42 dist-test-slave-49h3 krb5kdc[31711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079622, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:42.960006 31724 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:13:42.960889 31724 webserver.cc:460] Webserver started at http://127.8.16.126:39163/ using document root <none> and password file <none>
I20260807 05:13:42.961421 31724 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:42.961527 31724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:42.961738 31724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:42.963783 31724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "13371e05c6e74a7f943a175811f5466d"
format_stamp: "Formatted at 2026-08-07 05:13:42 on dist-test-slave-49h3"
server_key: "0d2bdb3e3bb00d5ddf3c7ba2bb23fec7"
server_key_iv: "a5f75b64bc345243dca066cd84c22727"
server_key_version: "encryptionkey@0"
I20260807 05:13:42.964340 31724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "13371e05c6e74a7f943a175811f5466d"
format_stamp: "Formatted at 2026-08-07 05:13:42 on dist-test-slave-49h3"
server_key: "0d2bdb3e3bb00d5ddf3c7ba2bb23fec7"
server_key_iv: "a5f75b64bc345243dca066cd84c22727"
server_key_version: "encryptionkey@0"
I20260807 05:13:42.968091 31724 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 05:13:42.970494 31739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:42.971662 31724 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:13:42.971783 31724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "13371e05c6e74a7f943a175811f5466d"
format_stamp: "Formatted at 2026-08-07 05:13:42 on dist-test-slave-49h3"
server_key: "0d2bdb3e3bb00d5ddf3c7ba2bb23fec7"
server_key_iv: "a5f75b64bc345243dca066cd84c22727"
server_key_version: "encryptionkey@0"
I20260807 05:13:42.971885 31724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:42.976468 31724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:42.979403 31724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:42.979575 31724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:42.986449 31724 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:39803
I20260807 05:13:42.986465 31795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:39803 every 8 connection(s)
I20260807 05:13:42.987396 31724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:42.990162 31796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:42.990235 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 31724
I20260807 05:13:42.990376 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:42.990675 8257 external_mini_cluster.cc:1428] Setting key 2701f114119a2777f51651889109d4ed
I20260807 05:13:42.995865 31799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:42.992014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34655 (local address 127.8.16.126:39803)
0807 05:13:42.992432 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:42.992447 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:42.992482 (+ 35us) server_negotiation.cc:407] Connection header received
0807 05:13:42.993247 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:42.993268 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:42.993587 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:42.993872 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:42.994699 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:42.994704 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:42.994716 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:42.994726 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:42.994788 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:42.994807 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:42.994910 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:42.995014 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 05:13:42.995151 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":113,"threads_started":1}
I20260807 05:13:42.996639 31796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13371e05c6e74a7f943a175811f5466d: Bootstrap starting.
I20260807 05:13:42.998647 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:13:42.999202 31796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13371e05c6e74a7f943a175811f5466d: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:43.000104 31796 log.cc:824] T 00000000000000000000000000000000 P 13371e05c6e74a7f943a175811f5466d: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:43.002416 31796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13371e05c6e74a7f943a175811f5466d: No bootstrap required, opened a new log
I20260807 05:13:43.002555 31799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:43.001025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37473 (local address 127.8.16.126:39803)
0807 05:13:43.001141 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:43.001146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:43.001316 (+ 170us) server_negotiation.cc:407] Connection header received
0807 05:13:43.001407 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:43.001410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:43.001467 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:43.001545 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:43.002181 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:43.002184 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:43.002187 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:13:43.002199 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:43.002227 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:13:43.002240 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:43.002326 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:43.002373 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 05:13:43.002409 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 05:13:43.006206 31796 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13371e05c6e74a7f943a175811f5466d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13371e05c6e74a7f943a175811f5466d" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 39803 } }
I20260807 05:13:43.006428 31796 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13371e05c6e74a7f943a175811f5466d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:13:43.006477 31796 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13371e05c6e74a7f943a175811f5466d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13371e05c6e74a7f943a175811f5466d, State: Initialized, Role: FOLLOWER
I20260807 05:13:43.006871 31796 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13371e05c6e74a7f943a175811f5466d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13371e05c6e74a7f943a175811f5466d" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 39803 } }
I20260807 05:13:43.006992 31796 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13371e05c6e74a7f943a175811f5466d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:13:43.007052 31796 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13371e05c6e74a7f943a175811f5466d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:13:43.007143 31796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13371e05c6e74a7f943a175811f5466d [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:43.008026 31796 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13371e05c6e74a7f943a175811f5466d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13371e05c6e74a7f943a175811f5466d" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 39803 } }
I20260807 05:13:43.008333 31796 leader_election.cc:302] T 00000000000000000000000000000000 P 13371e05c6e74a7f943a175811f5466d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13371e05c6e74a7f943a175811f5466d; no voters:
I20260807 05:13:43.008600 31796 leader_election.cc:288] T 00000000000000000000000000000000 P 13371e05c6e74a7f943a175811f5466d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:13:43.008697 31806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13371e05c6e74a7f943a175811f5466d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:43.008935 31806 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13371e05c6e74a7f943a175811f5466d [term 1 LEADER]: Becoming Leader. State: Replica: 13371e05c6e74a7f943a175811f5466d, State: Running, Role: LEADER
I20260807 05:13:43.009294 31806 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13371e05c6e74a7f943a175811f5466d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13371e05c6e74a7f943a175811f5466d" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 39803 } }
I20260807 05:13:43.009856 31796 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13371e05c6e74a7f943a175811f5466d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:43.010932 31807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13371e05c6e74a7f943a175811f5466d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13371e05c6e74a7f943a175811f5466d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13371e05c6e74a7f943a175811f5466d" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 39803 } } }
I20260807 05:13:43.010926 31808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13371e05c6e74a7f943a175811f5466d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13371e05c6e74a7f943a175811f5466d. Latest consensus state: current_term: 1 leader_uuid: "13371e05c6e74a7f943a175811f5466d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13371e05c6e74a7f943a175811f5466d" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 39803 } } }
I20260807 05:13:43.011067 31807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13371e05c6e74a7f943a175811f5466d [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:43.011080 31808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13371e05c6e74a7f943a175811f5466d [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:43.011461 31816 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:43.014056 31816 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:43.019732 31816 catalog_manager.cc:1347] Generated new cluster ID: a0902232651d4a0dbcdaa3e3e4f8be6b
I20260807 05:13:43.019798 31816 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:43.044070 31816 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:43.044415 31816 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:43.052474 31816 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13371e05c6e74a7f943a175811f5466d: Generated new TSK 0
I20260807 05:13:43.053323 31816 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:13:43.066510 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 31724
2026-08-07T05:13:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1343 ms)
[----------] 18 tests from AuthTokenIssuingTest (24235 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:13:43 dist-test-slave-49h3 krb5kdc[31828](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:13:43 dist-test-slave-49h3 krb5kdc[31828](info): set up 2 sockets
Aug 07 05:13:43 dist-test-slave-49h3 krb5kdc[31828](info): commencing operation
krb5kdc: starting...
W20260807 05:13:44.049785 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.962s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:13:44 dist-test-slave-49h3 krb5kdc[31828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:13:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:13:44.172326 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:43619
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:41689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:43619,127.8.16.125:34575,127.8.16.124:41529
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:44.280676 31841 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 05:13:44.280985 31841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:44.281019 31841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:44.281080 31841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:44.285081 31841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:44.285156 31841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:44.285181 31841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:44.285202 31841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:44.285220 31841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:44.290602 31841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:41689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:43619,127.8.16.125:34575,127.8.16.124:41529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:43619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:44.291709 31841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:44.292742 31841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:44.300940 31846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:44.300940 31849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:44.300979 31847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:44.301385 31841 server_base.cc:1034] running on GCE node
I20260807 05:13:44.301973 31841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:44.303072 31841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:44.304230 31841 hybrid_clock.cc:648] HybridClock initialized: now 1786079624304214 us; error 33 us; skew 500 ppm
Aug 07 05:13:44 dist-test-slave-49h3 krb5kdc[31828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079624, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:44.306955 31841 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:13:44.307858 31841 webserver.cc:460] Webserver started at http://127.8.16.126:39639/ using document root <none> and password file <none>
I20260807 05:13:44.308379 31841 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:44.308439 31841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:44.308635 31841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:44.310657 31841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b"
format_stamp: "Formatted at 2026-08-07 05:13:44 on dist-test-slave-49h3"
server_key: "dd6f187c9aec5d2966f780b9dd77f378"
server_key_iv: "c08980f8f7bb715e5c5fc64121061621"
server_key_version: "encryptionkey@0"
I20260807 05:13:44.311196 31841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b"
format_stamp: "Formatted at 2026-08-07 05:13:44 on dist-test-slave-49h3"
server_key: "dd6f187c9aec5d2966f780b9dd77f378"
server_key_iv: "c08980f8f7bb715e5c5fc64121061621"
server_key_version: "encryptionkey@0"
I20260807 05:13:44.314833 31841 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:13:44.317152 31856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:44.318326 31841 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:13:44.318437 31841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b"
format_stamp: "Formatted at 2026-08-07 05:13:44 on dist-test-slave-49h3"
server_key: "dd6f187c9aec5d2966f780b9dd77f378"
server_key_iv: "c08980f8f7bb715e5c5fc64121061621"
server_key_version: "encryptionkey@0"
I20260807 05:13:44.318535 31841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:44.328720 31841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:44.331337 31841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:44.331506 31841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:44.338227 31841 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:43619
I20260807 05:13:44.338518 31909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:43619 every 8 connection(s)
I20260807 05:13:44.339181 31841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:44.341928 31910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:44.345754 31910 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43619 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:13:44.348652 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 31841
I20260807 05:13:44.348790 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:44.349086 8257 external_mini_cluster.cc:1428] Setting key f7453256b0c677034cddaa93f75dd952
Aug 07 05:13:44 dist-test-slave-49h3 krb5kdc[31828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079624, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.16.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.125@KRBTEST.COM" created.
I20260807 05:13:44.361410 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.347969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:43619 (local address 127.0.0.1:54288)
0807 05:13:44.348563 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.348618 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:44.349662 (+ 1044us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:44.350355 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:44.350361 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:44.350805 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:44.351174 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:44.351190 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.352797 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.352800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:44.353621 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:44.353631 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.354487 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.354490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:44.354582 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.355125 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:44.355142 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:44.356691 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:44.358529 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:44.358538 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:44.358547 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:44.358783 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:44.359082 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:44.359089 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:44.359090 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:44.359237 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:44.359916 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:44.359921 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:44.360084 (+ 163us) client_negotiation.cc:769] Sending connection context
0807 05:13:44.360276 (+ 192us) client_negotiation.cc:240] Negotiation successful
0807 05:13:44.360565 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"spinlock_wait_cycles":19200,"thread_start_us":152,"threads_started":1}
I20260807 05:13:44.361415 31912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.347968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54288 (local address 127.8.16.126:43619)
0807 05:13:44.348409 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:44.348427 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:44.348735 (+ 308us) server_negotiation.cc:407] Connection header received
0807 05:13:44.349976 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:44.349980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:44.350159 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:44.350307 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:44.351352 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.352654 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.353764 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.354328 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.354497 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.356843 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:44.356848 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:44.356858 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:44.356880 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:44.358434 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.358869 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.358873 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.358874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.358975 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.359343 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.359345 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.359346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.359743 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:44.359851 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:44.360341 (+ 490us) server_negotiation.cc:299] Negotiation successful
0807 05:13:44.360663 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":89,"server-negotiator.queue_time_us":305,"spinlock_wait_cycles":896,"thread_start_us":135,"threads_started":1}
I20260807 05:13:44.363960 31910 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:13:44.364733 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.364465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:34575 (local address 127.0.0.1:39808)
0807 05:13:44.364580 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.364662 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.125:34575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 05:13:44.364964 31860 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.16.125:34575: connect: Connection refused (error 111)
I20260807 05:13:44.365947 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.365727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:34575 (local address 127.0.0.1:39816)
0807 05:13:44.365829 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.365883 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.125:34575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 05:13:44.366099 31910 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.125:34575: Network error: Client connection negotiation failed: client connection to 127.8.16.125:34575: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.125.keytab.
Entry for principal kudu/127.8.16.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.125.keytab.
I20260807 05:13:44.376688 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.125:34575
--webserver_interface=127.8.16.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.125.keytab
--principal=kudu/127.8.16.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:41689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:43619,127.8.16.125:34575,127.8.16.124:41529
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:13:44.415377 31910 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } attempt: 1
I20260807 05:13:44.416283 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.415982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:34575 (local address 127.0.0.1:39818)
0807 05:13:44.416136 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.416201 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.125:34575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 05:13:44.416955 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.416776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:34575 (local address 127.0.0.1:39826)
0807 05:13:44.416858 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.416900 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.125:34575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 05:13:44.417070 31910 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.125:34575: Network error: Client connection negotiation failed: client connection to 127.8.16.125:34575: connect: Connection refused (error 111)
I20260807 05:13:44.485239 31910 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } attempt: 2
W20260807 05:13:44.485622 31918 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 05:13:44.485913 31918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:44.485965 31918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:44.486022 31918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 05:13:44.486104 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.485787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:34575 (local address 127.0.0.1:39840)
0807 05:13:44.485930 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.486001 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.125:34575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 05:13:44.486793 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.486556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:34575 (local address 127.0.0.1:39842)
0807 05:13:44.486670 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.486726 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.125:34575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 05:13:44.486934 31910 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.125:34575: Network error: Client connection negotiation failed: client connection to 127.8.16.125:34575: connect: Connection refused (error 111)
W20260807 05:13:44.490283 31918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:44.490334 31918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:44.490352 31918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:44.490370 31918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:44.490396 31918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:44.495678 31918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:41689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:43619,127.8.16.125:34575,127.8.16.124:41529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.125.keytab
--principal=kudu/127.8.16.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.125:34575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.16.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:44.496695 31918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:44.497689 31918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:44.505779 31927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:44.505786 31925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:44.505848 31918 server_base.cc:1034] running on GCE node
W20260807 05:13:44.505786 31924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:44.506515 31918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:44.507520 31918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:44.508675 31918 hybrid_clock.cc:648] HybridClock initialized: now 1786079624508654 us; error 32 us; skew 500 ppm
Aug 07 05:13:44 dist-test-slave-49h3 krb5kdc[31828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079624, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:44.511194 31918 init.cc:377] Logged in from keytab as kudu/127.8.16.125@KRBTEST.COM (short username kudu)
I20260807 05:13:44.512014 31918 webserver.cc:460] Webserver started at http://127.8.16.125:34845/ using document root <none> and password file <none>
I20260807 05:13:44.512535 31918 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:44.512594 31918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:44.512789 31918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:44.514779 31918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-1/data/instance:
uuid: "32c3f22839954ec4bb46f17331761b4d"
format_stamp: "Formatted at 2026-08-07 05:13:44 on dist-test-slave-49h3"
server_key: "b677ec78821b6efa4c6caaa82dc29438"
server_key_iv: "49aa7935b32210f8dbb19cd5f7e9df65"
server_key_version: "encryptionkey@0"
I20260807 05:13:44.515287 31918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-1/wal/instance:
uuid: "32c3f22839954ec4bb46f17331761b4d"
format_stamp: "Formatted at 2026-08-07 05:13:44 on dist-test-slave-49h3"
server_key: "b677ec78821b6efa4c6caaa82dc29438"
server_key_iv: "49aa7935b32210f8dbb19cd5f7e9df65"
server_key_version: "encryptionkey@0"
I20260807 05:13:44.518795 31918 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:13:44.520913 31935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:44.522068 31918 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:13:44.522213 31918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-1/wal
uuid: "32c3f22839954ec4bb46f17331761b4d"
format_stamp: "Formatted at 2026-08-07 05:13:44 on dist-test-slave-49h3"
server_key: "b677ec78821b6efa4c6caaa82dc29438"
server_key_iv: "49aa7935b32210f8dbb19cd5f7e9df65"
server_key_version: "encryptionkey@0"
I20260807 05:13:44.522305 31918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:44.545766 31918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:44.548921 31918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:44.549100 31918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:44.555608 31918 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.125:34575
I20260807 05:13:44.555857 31987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.125:34575 every 8 connection(s)
I20260807 05:13:44.556543 31918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-1/data/info.pb
I20260807 05:13:44.559163 31988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:44.562834 31988 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43619 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:13:44.563352 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 31918
I20260807 05:13:44.563467 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-1/wal/instance
I20260807 05:13:44.563730 8257 external_mini_cluster.cc:1428] Setting key 9c5dc652a83144d06646808207e8be12
WARNING: no policy specified for kudu/127.8.16.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.124@KRBTEST.COM" created.
Aug 07 05:13:44 dist-test-slave-49h3 krb5kdc[31828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079624, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.125@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:13:44.578074 31910 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } attempt: 3
I20260807 05:13:44.579350 31912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.564968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54302 (local address 127.8.16.126:43619)
0807 05:13:44.565128 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:44.565132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:44.566014 (+ 882us) server_negotiation.cc:407] Connection header received
0807 05:13:44.566940 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:44.566944 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:44.567000 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:44.567072 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:44.568184 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.568711 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.569870 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.570118 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.570211 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.574960 (+ 4749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:44.574963 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:44.574965 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:44.574994 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:44.576523 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.577231 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.577235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.577237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.577395 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.577921 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.577925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.577927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.578191 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:44.578283 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:44.579034 (+ 751us) server_negotiation.cc:299] Negotiation successful
0807 05:13:44.579170 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 05:13:44.580217 31991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.565343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:43619 (local address 127.0.0.1:54302)
0807 05:13:44.565845 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.565897 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:44.566742 (+ 845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:44.567240 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:44.567246 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:44.567585 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:44.568043 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:44.568055 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.568835 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.568838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:44.569753 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:44.569764 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.570275 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.570280 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:44.570435 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.571075 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:44.571097 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:44.574772 (+ 3675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:44.576666 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:44.576674 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:44.576694 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:44.577087 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:44.577553 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:44.577557 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:44.577559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:44.577784 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:44.578346 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:44.578351 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:44.578715 (+ 364us) client_negotiation.cc:769] Sending connection context
0807 05:13:44.578968 (+ 253us) client_negotiation.cc:240] Negotiation successful
0807 05:13:44.579307 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":129,"threads_started":1}
I20260807 05:13:44.581692 31988 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 05:13:44 dist-test-slave-49h3 krb5kdc[31828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079624, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for kudu/127.8.16.125@KRBTEST.COM
Aug 07 05:13:44 dist-test-slave-49h3 krb5kdc[31828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079624, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.125@KRBTEST.COM for kudu/127.8.16.125@KRBTEST.COM
I20260807 05:13:44.589771 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.578600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:34575 (local address 127.0.0.1:39852)
0807 05:13:44.578758 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.578775 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:44.578854 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:44.580201 (+ 1347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:44.580204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:44.580389 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:44.580506 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:44.580514 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.582498 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.582502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:44.583314 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:44.583327 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.583907 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.583911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:44.584022 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.584450 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:44.584464 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:44.586088 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:44.588011 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:44.588015 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:44.588017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:44.588239 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:44.588560 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:44.588563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:44.588565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:44.588649 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:44.589161 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:44.589163 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:44.589263 (+ 100us) client_negotiation.cc:769] Sending connection context
0807 05:13:44.589400 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 05:13:44.589588 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 05:13:44.589960 31993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.578918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39852 (local address 127.8.16.125:34575)
0807 05:13:44.579729 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:44.579742 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:44.579766 (+ 24us) server_negotiation.cc:407] Connection header received
0807 05:13:44.579811 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:44.579817 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:44.580042 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:44.580194 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:44.580855 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.582349 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.583431 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.583796 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.583903 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.586220 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:44.586231 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:44.586247 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:44.586273 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:44.587886 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.588381 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.588387 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.588389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.588470 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.588730 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.588734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.588735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.589044 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:44.589176 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:44.589675 (+ 499us) server_negotiation.cc:299] Negotiation successful
0807 05:13:44.589802 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":94,"threads_started":1}
I20260807 05:13:44.590364 31991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.582175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:34575 (local address 127.0.0.1:39868)
0807 05:13:44.582453 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.582466 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:44.582540 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:44.582816 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:44.582818 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:44.582949 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:44.583022 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:44.583027 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.583742 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.583745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:44.584450 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:44.584461 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.584950 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.584953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:44.585047 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.585493 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:44.585507 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:44.586722 (+ 1215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:44.588955 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:44.588959 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:44.588961 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:44.589174 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:44.589418 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:44.589421 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:44.589423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:44.589538 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:44.589940 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:44.589942 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:44.590020 (+ 78us) client_negotiation.cc:769] Sending connection context
0807 05:13:44.590111 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 05:13:44.590248 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216}
I20260807 05:13:44.590762 31994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.582196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39868 (local address 127.8.16.125:34575)
0807 05:13:44.582487 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:44.582494 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:44.582506 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:13:44.582604 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:44.582607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:44.582642 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:44.582710 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:44.583122 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.583642 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.584562 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.584835 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.584912 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.586866 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:44.586870 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:44.586871 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:44.586895 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:44.588819 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.589258 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.589261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.589262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.589348 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.589620 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.589623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.589625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.589811 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:44.589904 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:44.590537 (+ 633us) server_negotiation.cc:299] Negotiation successful
0807 05:13:44.590644 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":119,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.124.keytab.
Entry for principal kudu/127.8.16.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.124.keytab.
I20260807 05:13:44.592141 31988 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:13:44.592495 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.124:41529
--webserver_interface=127.8.16.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.124.keytab
--principal=kudu/127.8.16.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:41689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:43619,127.8.16.125:34575,127.8.16.124:41529
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:13:44.592794 31991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.592553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:41529 (local address 127.0.0.1:50412)
0807 05:13:44.592662 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.592742 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:41529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 05:13:44.592962 31937 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.16.124:41529: connect: Connection refused (error 111)
I20260807 05:13:44.593824 31991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.593618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:41529 (local address 127.0.0.1:50414)
0807 05:13:44.593732 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.593776 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:41529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 05:13:44.593940 31988 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.124:41529: Network error: Client connection negotiation failed: client connection to 127.8.16.124:41529: connect: Connection refused (error 111)
I20260807 05:13:44.594223 31910 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:13:44.595382 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.595158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:41529 (local address 127.0.0.1:50422)
0807 05:13:44.595277 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.595322 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:41529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 05:13:44.595978 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.595770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:41529 (local address 127.0.0.1:50436)
0807 05:13:44.595894 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.595932 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:41529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260807 05:13:44.596082 31910 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.124:41529: Network error: Client connection negotiation failed: client connection to 127.8.16.124:41529: connect: Connection refused (error 111)
I20260807 05:13:44.627242 31910 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } attempt: 1
I20260807 05:13:44.628132 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.627825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:41529 (local address 127.0.0.1:50444)
0807 05:13:44.627985 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.628048 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:41529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 05:13:44.628782 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.628600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:41529 (local address 127.0.0.1:50450)
0807 05:13:44.628680 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.628723 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:41529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 05:13:44.628899 31910 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.124:41529: Network error: Client connection negotiation failed: client connection to 127.8.16.124:41529: connect: Connection refused (error 111)
I20260807 05:13:44.643225 31988 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } attempt: 1
I20260807 05:13:44.644158 31991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.643813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:41529 (local address 127.0.0.1:50452)
0807 05:13:44.643976 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.644053 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:41529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 05:13:44.644881 31991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.644653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:41529 (local address 127.0.0.1:50460)
0807 05:13:44.644767 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.644818 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:41529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 05:13:44.645009 31988 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.124:41529: Network error: Client connection negotiation failed: client connection to 127.8.16.124:41529: connect: Connection refused (error 111)
W20260807 05:13:44.703174 31996 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 05:13:44.703446 31996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:44.703471 31996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:44.703531 31996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 05:13:44.704051 31910 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } attempt: 2
I20260807 05:13:44.704919 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.704593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:41529 (local address 127.0.0.1:50472)
0807 05:13:44.704768 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.704838 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:41529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 05:13:44.705615 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.705375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:41529 (local address 127.0.0.1:50478)
0807 05:13:44.705516 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.705562 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:41529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 05:13:44.705732 31910 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.124:41529: Network error: Client connection negotiation failed: client connection to 127.8.16.124:41529: connect: Connection refused (error 111)
W20260807 05:13:44.707976 31996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:44.708030 31996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:44.708050 31996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:44.708067 31996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:44.708094 31996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:44.713166 31988 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } attempt: 2
I20260807 05:13:44.713531 31996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:41689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:43619,127.8.16.125:34575,127.8.16.124:41529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.124.keytab
--principal=kudu/127.8.16.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.124:41529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.16.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:44.713970 31991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.713655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:41529 (local address 127.0.0.1:50484)
0807 05:13:44.713844 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.713898 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:41529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260807 05:13:44.714607 31991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.714412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:41529 (local address 127.0.0.1:50500)
0807 05:13:44.714509 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.714551 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:41529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 05:13:44.714653 31996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
W20260807 05:13:44.714723 31988 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.124:41529: Network error: Client connection negotiation failed: client connection to 127.8.16.124:41529: connect: Connection refused (error 111)
I20260807 05:13:44.715687 31996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:44.723739 32001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:44.723728 32002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:44.723865 32004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:44.724117 31996 server_base.cc:1034] running on GCE node
I20260807 05:13:44.724819 31996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:44.726003 31996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:44.727178 31996 hybrid_clock.cc:648] HybridClock initialized: now 1786079624727167 us; error 37 us; skew 500 ppm
Aug 07 05:13:44 dist-test-slave-49h3 krb5kdc[31828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079624, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:44.729907 31996 init.cc:377] Logged in from keytab as kudu/127.8.16.124@KRBTEST.COM (short username kudu)
I20260807 05:13:44.730741 31996 webserver.cc:460] Webserver started at http://127.8.16.124:44359/ using document root <none> and password file <none>
I20260807 05:13:44.731251 31996 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:44.731310 31996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:44.731482 31996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:44.733527 31996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-2/data/instance:
uuid: "156f0048d47f45a5962cfa3109df8872"
format_stamp: "Formatted at 2026-08-07 05:13:44 on dist-test-slave-49h3"
server_key: "a1d4fee7a9d62ce012301612b719b537"
server_key_iv: "60ed1ae8a02a01dd7f4ba92b0c996cb0"
server_key_version: "encryptionkey@0"
I20260807 05:13:44.734077 31996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-2/wal/instance:
uuid: "156f0048d47f45a5962cfa3109df8872"
format_stamp: "Formatted at 2026-08-07 05:13:44 on dist-test-slave-49h3"
server_key: "a1d4fee7a9d62ce012301612b719b537"
server_key_iv: "60ed1ae8a02a01dd7f4ba92b0c996cb0"
server_key_version: "encryptionkey@0"
I20260807 05:13:44.737699 31996 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260807 05:13:44.740052 32011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:44.741283 31996 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:13:44.741392 31996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-2/wal
uuid: "156f0048d47f45a5962cfa3109df8872"
format_stamp: "Formatted at 2026-08-07 05:13:44 on dist-test-slave-49h3"
server_key: "a1d4fee7a9d62ce012301612b719b537"
server_key_iv: "60ed1ae8a02a01dd7f4ba92b0c996cb0"
server_key_version: "encryptionkey@0"
I20260807 05:13:44.741537 31996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:44.763070 31996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:44.766366 31996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:44.766556 31996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:44.773185 31996 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.124:41529
I20260807 05:13:44.773483 32066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.124:41529 every 8 connection(s)
I20260807 05:13:44.774191 31996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-2/data/info.pb
I20260807 05:13:44.776866 32067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:44.779819 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 31996
I20260807 05:13:44.779924 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079354071073-8257-0/minicluster-data/master-2/wal/instance
I20260807 05:13:44.780164 8257 external_mini_cluster.cc:1428] Setting key 8bfed4cd83fc06ca381a3c389d339f1d
I20260807 05:13:44.780678 32067 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43619 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 05:13:44 dist-test-slave-49h3 krb5kdc[31828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079624, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.124@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
Aug 07 05:13:44 dist-test-slave-49h3 krb5kdc[31828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.124@KRBTEST.COM
I20260807 05:13:44.793074 31912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.782679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54304 (local address 127.8.16.126:43619)
0807 05:13:44.782812 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:44.782816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:44.783032 (+ 216us) server_negotiation.cc:407] Connection header received
0807 05:13:44.783312 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:44.783315 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:44.783362 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:44.783437 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:44.784408 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.785178 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.786532 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.786861 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.786965 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.789486 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:44.789490 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:44.789493 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:44.789523 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:44.791003 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.791504 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.791507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.791509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.791600 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.791945 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.791948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.791950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.792189 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:44.792270 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:44.792784 (+ 514us) server_negotiation.cc:299] Negotiation successful
0807 05:13:44.792908 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 05:13:44.793924 32070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.781560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50502 (local address 127.8.16.124:41529)
0807 05:13:44.781973 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:44.781990 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:44.782027 (+ 37us) server_negotiation.cc:407] Connection header received
0807 05:13:44.782771 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:44.782792 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:44.783146 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:44.783352 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:44.784243 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.785428 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.786605 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.787128 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.787208 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.789894 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:44.789901 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:44.789914 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:44.789938 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:44.791500 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.791943 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.791948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.791950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.792044 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.792281 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.792284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.792286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.792523 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:44.792667 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:44.792917 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 05:13:44.793083 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":267,"thread_start_us":108,"threads_started":1}
I20260807 05:13:44.793922 32071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.782571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:43619 (local address 127.0.0.1:54304)
0807 05:13:44.782894 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.782942 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:44.783140 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:44.783502 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:44.783507 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:44.783941 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:44.784281 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:44.784294 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.785272 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.785274 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:44.786422 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:44.786433 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.786990 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.786993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:44.787137 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.787662 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:44.787683 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:44.789320 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:44.791101 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:44.791108 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:44.791116 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:44.791415 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:44.791692 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:44.791695 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:44.791697 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:44.791865 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:44.792291 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:44.792297 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:44.792541 (+ 244us) client_negotiation.cc:769] Sending connection context
0807 05:13:44.792703 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 05:13:44.793028 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":4,"spinlock_wait_cycles":896,"thread_start_us":66,"threads_started":1}
I20260807 05:13:44.795382 32067 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 05:13:44 dist-test-slave-49h3 krb5kdc[31828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079624, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.124@KRBTEST.COM for kudu/127.8.16.125@KRBTEST.COM
Aug 07 05:13:44 dist-test-slave-49h3 krb5kdc[31828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:13:44.803373 32071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.795846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:34575 (local address 127.0.0.1:39870)
0807 05:13:44.795999 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.796013 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:44.796078 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:44.796449 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:44.796452 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:44.796621 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:44.796729 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:44.796735 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.797472 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.797475 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:44.797986 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:44.797993 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.798441 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.798445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:44.798544 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.798969 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:44.798983 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:44.800204 (+ 1221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:44.801971 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:44.801974 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:44.801975 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:44.802152 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:44.802430 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:44.802433 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:44.802435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:44.802540 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:44.802934 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:44.802937 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:44.803037 (+ 100us) client_negotiation.cc:769] Sending connection context
0807 05:13:44.803137 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 05:13:44.803251 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260807 05:13:44.803481 31994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.795941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39870 (local address 127.8.16.125:34575)
0807 05:13:44.796048 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:44.796051 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:44.796064 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:13:44.796212 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:44.796216 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:44.796272 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:44.796317 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:44.796841 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.797355 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.798090 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.798343 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.798435 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.800333 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:44.800337 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:44.800339 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:44.800363 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:44.801870 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.802250 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.802254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.802256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.802342 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.802626 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.802628 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.802629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.802850 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:44.802939 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:44.803198 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 05:13:44.803322 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 05:13:44.804180 31912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.796322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54308 (local address 127.8.16.126:43619)
0807 05:13:44.796439 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:44.796443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:44.796454 (+ 11us) server_negotiation.cc:407] Connection header received
0807 05:13:44.796482 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:44.796485 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:44.796524 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:44.796596 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:44.797137 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.797674 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.798400 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.798597 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.798682 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.800938 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:44.800942 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:44.800945 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:44.800969 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:44.802324 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.802760 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.802763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.802765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.802843 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.803093 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.803095 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.803096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.803314 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:44.803849 (+ 535us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:44.803928 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 05:13:44.804043 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:13:44.804217 32067 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:13:44.804857 31910 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } attempt: 3
I20260807 05:13:44.805861 31988 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } attempt: 3
Aug 07 05:13:44 dist-test-slave-49h3 krb5kdc[31828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079624, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.124@KRBTEST.COM for kudu/127.8.16.124@KRBTEST.COM
Aug 07 05:13:44 dist-test-slave-49h3 krb5kdc[31828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079624, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for kudu/127.8.16.124@KRBTEST.COM
Aug 07 05:13:44 dist-test-slave-49h3 krb5kdc[31828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.125@KRBTEST.COM
Aug 07 05:13:44 dist-test-slave-49h3 krb5kdc[31828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079624, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.125@KRBTEST.COM for kudu/127.8.16.124@KRBTEST.COM
I20260807 05:13:44.813958 32071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.804692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:41529 (local address 127.0.0.1:50514)
0807 05:13:44.804862 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.804876 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:44.804951 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:44.805210 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:44.805213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:44.805364 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:44.805475 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:44.805482 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.806637 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.806640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:44.807448 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:44.807457 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.808089 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.808092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:44.808196 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.808689 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:44.808703 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:44.810728 (+ 2025us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:44.812447 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:44.812450 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:44.812452 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:44.812678 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:44.812939 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:44.812942 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:44.812944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:44.813029 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:44.813486 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:44.813489 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:44.813573 (+ 84us) client_negotiation.cc:769] Sending connection context
0807 05:13:44.813679 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 05:13:44.813804 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":24}
I20260807 05:13:44.814429 32070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.804779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50514 (local address 127.8.16.124:41529)
0807 05:13:44.804865 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:44.804869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:44.804914 (+ 45us) server_negotiation.cc:407] Connection header received
0807 05:13:44.805035 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:44.805037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:44.805069 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:44.805121 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:44.805583 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.806532 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.807558 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.807902 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.807972 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.810843 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:44.810846 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:44.810849 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:44.810870 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:44.812344 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.812776 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.812779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.812781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.812854 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.813112 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.813114 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.813116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.813378 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:44.813486 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:44.814163 (+ 677us) server_negotiation.cc:299] Negotiation successful
0807 05:13:44.814291 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":23}
I20260807 05:13:44.815348 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.805242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:41529 (local address 127.0.0.1:50526)
0807 05:13:44.805402 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.805415 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:44.805522 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:44.805899 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:44.805903 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:44.806079 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:44.806180 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:44.806187 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.807627 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.807631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:44.808365 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:44.808375 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.808952 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.808956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:44.809043 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.809490 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:44.809503 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:44.810713 (+ 1210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:44.813699 (+ 2986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:44.813702 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:44.813704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:44.813944 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:44.814397 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:44.814400 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:44.814402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:44.814491 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:44.814909 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:44.814913 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:44.815011 (+ 98us) client_negotiation.cc:769] Sending connection context
0807 05:13:44.815092 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 05:13:44.815203 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260807 05:13:44.815946 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.805293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50526 (local address 127.8.16.124:41529)
0807 05:13:44.805705 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:44.805712 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:44.805722 (+ 10us) server_negotiation.cc:407] Connection header received
0807 05:13:44.805751 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:44.805753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:44.805790 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:44.805871 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:44.806352 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.807518 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.808497 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.808831 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.808934 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.811480 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:44.811484 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:44.811486 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:44.811510 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:44.813575 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.814179 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.814182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.814184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.814286 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.814601 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.814604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.814606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.814798 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:44.815613 (+ 815us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:44.815682 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 05:13:44.815790 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":357,"thread_start_us":156,"threads_started":1}
I20260807 05:13:44.816383 31991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.806234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:41529 (local address 127.0.0.1:50536)
0807 05:13:44.806400 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:44.806413 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:44.806477 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:44.807155 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:44.807158 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:44.807329 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:44.807411 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:44.807417 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.808496 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.808499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:44.809144 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:44.809154 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.810038 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.810041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:44.810121 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.810876 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:44.810891 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:44.812728 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:44.814974 (+ 2246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:44.814977 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:44.814979 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:44.815214 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:44.815500 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:44.815503 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:44.815505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:44.815596 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:44.816004 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:44.816008 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:44.816104 (+ 96us) client_negotiation.cc:769] Sending connection context
0807 05:13:44.816171 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 05:13:44.816272 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":1}
I20260807 05:13:44.817313 32073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.806644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50536 (local address 127.8.16.124:41529)
0807 05:13:44.806949 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:44.806957 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:44.806969 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:13:44.806999 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:44.807002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:44.807036 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:44.807501 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:44.807621 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.808394 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.809570 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.809918 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.810016 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.812859 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:44.812863 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:44.812865 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:44.812889 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:44.814875 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.815304 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.815307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.815309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.815391 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.815699 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.815701 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.815703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.815893 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:44.816980 (+ 1087us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:44.817061 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 05:13:44.817166 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":247,"thread_start_us":106,"threads_started":1}
I20260807 05:13:44.817590 31994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.805511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39882 (local address 127.8.16.125:34575)
0807 05:13:44.805616 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:44.805620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:44.805691 (+ 71us) server_negotiation.cc:407] Connection header received
0807 05:13:44.805813 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:44.805816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:44.805854 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:44.805916 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:44.806676 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.807354 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.808432 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.809353 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.809437 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.813012 (+ 3575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:44.813015 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:44.813017 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:44.813040 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:44.815718 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.816166 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.816183 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.816184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.816269 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.816580 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.816583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.816585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.816772 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:44.816941 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:44.817310 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 05:13:44.817417 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":622,"server-negotiator.queue_time_us":33}
I20260807 05:13:44.819231 31910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b: Bootstrap starting.
I20260807 05:13:44.819510 31988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32c3f22839954ec4bb46f17331761b4d: Bootstrap starting.
I20260807 05:13:44.821386 31910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:44.821388 31988 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32c3f22839954ec4bb46f17331761b4d: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:44.822201 31988 log.cc:824] T 00000000000000000000000000000000 P 32c3f22839954ec4bb46f17331761b4d: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:44.822216 31910 log.cc:824] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:44.823527 32067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 156f0048d47f45a5962cfa3109df8872: Bootstrap starting.
I20260807 05:13:44.824386 31910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b: No bootstrap required, opened a new log
I20260807 05:13:44.824394 31988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32c3f22839954ec4bb46f17331761b4d: No bootstrap required, opened a new log
I20260807 05:13:44.826215 32067 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 156f0048d47f45a5962cfa3109df8872: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:44.826952 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:13:44.827039 32067 log.cc:824] T 00000000000000000000000000000000 P 156f0048d47f45a5962cfa3109df8872: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:44.828318 31988 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32c3f22839954ec4bb46f17331761b4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43619 } } peers { permanent_uuid: "32c3f22839954ec4bb46f17331761b4d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } } peers { permanent_uuid: "156f0048d47f45a5962cfa3109df8872" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } }
I20260807 05:13:44.828315 31910 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43619 } } peers { permanent_uuid: "32c3f22839954ec4bb46f17331761b4d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } } peers { permanent_uuid: "156f0048d47f45a5962cfa3109df8872" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } }
I20260807 05:13:44.828537 31988 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32c3f22839954ec4bb46f17331761b4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32c3f22839954ec4bb46f17331761b4d, State: Initialized, Role: FOLLOWER
I20260807 05:13:44.828541 31910 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6c9ea3ac8d14212b0d784fdd4142f8b, State: Initialized, Role: FOLLOWER
I20260807 05:13:44.828508 31883 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54308
I20260807 05:13:44.828859 31910 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43619 } } peers { permanent_uuid: "32c3f22839954ec4bb46f17331761b4d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } } peers { permanent_uuid: "156f0048d47f45a5962cfa3109df8872" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } }
I20260807 05:13:44.828859 31988 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32c3f22839954ec4bb46f17331761b4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43619 } } peers { permanent_uuid: "32c3f22839954ec4bb46f17331761b4d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } } peers { permanent_uuid: "156f0048d47f45a5962cfa3109df8872" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } }
I20260807 05:13:44.829234 32067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 156f0048d47f45a5962cfa3109df8872: No bootstrap required, opened a new log
I20260807 05:13:44.829519 32079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32c3f22839954ec4bb46f17331761b4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43619 } } peers { permanent_uuid: "32c3f22839954ec4bb46f17331761b4d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } } peers { permanent_uuid: "156f0048d47f45a5962cfa3109df8872" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } } }
I20260807 05:13:44.829849 32079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32c3f22839954ec4bb46f17331761b4d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:13:44.830310 31988 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32c3f22839954ec4bb46f17331761b4d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:44.830350 31910 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:44.830384 31883 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 05:13:44.830485 31883 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:44.831789 31883 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43619 } } peers { permanent_uuid: "32c3f22839954ec4bb46f17331761b4d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } } peers { permanent_uuid: "156f0048d47f45a5962cfa3109df8872" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } }
I20260807 05:13:44.832736 31883 leader_election.cc:288] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b [CANDIDATE]: Term 1 election: Requested vote from peers 32c3f22839954ec4bb46f17331761b4d (127.8.16.125:34575), 156f0048d47f45a5962cfa3109df8872 (127.8.16.124:41529)
I20260807 05:13:44.833981 32067 raft_consensus.cc:348] T 00000000000000000000000000000000 P 156f0048d47f45a5962cfa3109df8872 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43619 } } peers { permanent_uuid: "32c3f22839954ec4bb46f17331761b4d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } } peers { permanent_uuid: "156f0048d47f45a5962cfa3109df8872" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } }
I20260807 05:13:44.834223 32067 raft_consensus.cc:729] T 00000000000000000000000000000000 P 156f0048d47f45a5962cfa3109df8872 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 156f0048d47f45a5962cfa3109df8872, State: Initialized, Role: FOLLOWER
I20260807 05:13:44.834179 31962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "32c3f22839954ec4bb46f17331761b4d"
I20260807 05:13:44.834376 31962 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32c3f22839954ec4bb46f17331761b4d [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:44.834547 32067 consensus_queue.cc:260] T 00000000000000000000000000000000 P 156f0048d47f45a5962cfa3109df8872 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43619 } } peers { permanent_uuid: "32c3f22839954ec4bb46f17331761b4d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } } peers { permanent_uuid: "156f0048d47f45a5962cfa3109df8872" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } }
W20260807 05:13:44.835572 32103 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 32c3f22839954ec4bb46f17331761b4d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:13:44.835649 32103 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:13:44.835695 31962 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 32c3f22839954ec4bb46f17331761b4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6c9ea3ac8d14212b0d784fdd4142f8b in term 1.
I20260807 05:13:44.835788 32067 sys_catalog.cc:564] T 00000000000000000000000000000000 P 156f0048d47f45a5962cfa3109df8872 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:44.836066 32105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 156f0048d47f45a5962cfa3109df8872 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43619 } } peers { permanent_uuid: "32c3f22839954ec4bb46f17331761b4d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } } peers { permanent_uuid: "156f0048d47f45a5962cfa3109df8872" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } } }
I20260807 05:13:44.836179 32105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 156f0048d47f45a5962cfa3109df8872 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:13:44.836916 32041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "156f0048d47f45a5962cfa3109df8872"
I20260807 05:13:44.837116 32041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 156f0048d47f45a5962cfa3109df8872 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:44.837783 32078 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43619 } } peers { permanent_uuid: "32c3f22839954ec4bb46f17331761b4d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } } peers { permanent_uuid: "156f0048d47f45a5962cfa3109df8872" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } } }
I20260807 05:13:44.837900 32078 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b [sys.catalog]: This master's current role is: FOLLOWER
W20260807 05:13:44.833408 31862 tablet.cc:2369] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:13:44.836129 31860 leader_election.cc:302] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32c3f22839954ec4bb46f17331761b4d, d6c9ea3ac8d14212b0d784fdd4142f8b; no voters:
I20260807 05:13:44.838440 32041 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 156f0048d47f45a5962cfa3109df8872 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6c9ea3ac8d14212b0d784fdd4142f8b in term 1.
I20260807 05:13:44.839175 32078 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:44.840003 32078 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b [term 1 LEADER]: Becoming Leader. State: Replica: d6c9ea3ac8d14212b0d784fdd4142f8b, State: Running, Role: LEADER
I20260807 05:13:44.840341 32078 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43619 } } peers { permanent_uuid: "32c3f22839954ec4bb46f17331761b4d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } } peers { permanent_uuid: "156f0048d47f45a5962cfa3109df8872" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } }
I20260807 05:13:44.842154 32078 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6c9ea3ac8d14212b0d784fdd4142f8b. Latest consensus state: current_term: 1 leader_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43619 } } peers { permanent_uuid: "32c3f22839954ec4bb46f17331761b4d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } } peers { permanent_uuid: "156f0048d47f45a5962cfa3109df8872" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } } }
I20260807 05:13:44.842274 32078 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:44.845588 31918 master_runner.cc:263] No leader master found from master 32c3f22839954ec4bb46f17331761b4d
I20260807 05:13:44.845669 31918 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 05:13:44.847406 31841 master_runner.cc:263] No leader master found from master d6c9ea3ac8d14212b0d784fdd4142f8b
I20260807 05:13:44.847473 31841 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 05:13:44.849083 32099 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:13:44.849187 32099 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:13:44.849350 32129 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:44.850267 32129 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260807 05:13:44.852408 32124 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 156f0048d47f45a5962cfa3109df8872: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:13:44.852499 32124 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:13:44.853631 31912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.838772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54312 (local address 127.8.16.126:43619)
0807 05:13:44.838905 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:44.838910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:44.839379 (+ 469us) server_negotiation.cc:407] Connection header received
0807 05:13:44.839570 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:44.839573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:44.839621 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:44.839674 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:44.842534 (+ 2860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.843319 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.844922 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.845267 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.845346 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.848396 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:44.848400 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:44.848403 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:44.848431 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:44.850115 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.852175 (+ 2060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.852180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.852182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.852274 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.852579 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.852583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.852584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.852812 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:44.852920 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:44.853313 (+ 393us) server_negotiation.cc:299] Negotiation successful
0807 05:13:44.853468 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 05:13:44.861039 31994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.839592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39888 (local address 127.8.16.125:34575)
0807 05:13:44.839806 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:44.839810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:44.845926 (+ 6116us) server_negotiation.cc:407] Connection header received
0807 05:13:44.846845 (+ 919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:44.846848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:44.846900 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:44.846953 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:44.848979 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.850473 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.854091 (+ 3618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.854402 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.854480 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.857370 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:44.857375 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:44.857378 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:44.857413 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:44.859064 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.859595 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.859599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.859601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.859692 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.860006 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.860009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.860011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.860239 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:44.860379 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:44.860742 (+ 363us) server_negotiation.cc:299] Negotiation successful
0807 05:13:44.860869 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260807 05:13:44.864565 32073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:44.839493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50542 (local address 127.8.16.124:41529)
0807 05:13:44.839586 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:44.839590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:44.844776 (+ 5186us) server_negotiation.cc:407] Connection header received
0807 05:13:44.844884 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:44.844886 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:44.844934 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:44.845014 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:44.847689 (+ 2675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.848378 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.851180 (+ 2802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:44.851535 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:44.851635 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:44.854483 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:44.854487 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:44.854489 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:44.854514 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:44.856289 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.863134 (+ 6845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.863138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.863140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.863244 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:44.863565 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:44.863569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:44.863570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:44.863793 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:44.863907 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:44.864273 (+ 366us) server_negotiation.cc:299] Negotiation successful
0807 05:13:44.864407 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 05:13:44.870119 32041 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 156f0048d47f45a5962cfa3109df8872 [term 1 FOLLOWER]: Refusing update from remote peer d6c9ea3ac8d14212b0d784fdd4142f8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:13:44.872512 32126 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "156f0048d47f45a5962cfa3109df8872" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:13:44.876695 32105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 156f0048d47f45a5962cfa3109df8872 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6c9ea3ac8d14212b0d784fdd4142f8b. Latest consensus state: current_term: 1 leader_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43619 } } peers { permanent_uuid: "32c3f22839954ec4bb46f17331761b4d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } } peers { permanent_uuid: "156f0048d47f45a5962cfa3109df8872" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } } }
I20260807 05:13:44.876823 32105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 156f0048d47f45a5962cfa3109df8872 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:13:44.878818 31962 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 32c3f22839954ec4bb46f17331761b4d [term 1 FOLLOWER]: Refusing update from remote peer d6c9ea3ac8d14212b0d784fdd4142f8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:13:44.879411 32126 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "32c3f22839954ec4bb46f17331761b4d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:13:44.883005 32079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32c3f22839954ec4bb46f17331761b4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6c9ea3ac8d14212b0d784fdd4142f8b. Latest consensus state: current_term: 1 leader_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43619 } } peers { permanent_uuid: "32c3f22839954ec4bb46f17331761b4d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } } peers { permanent_uuid: "156f0048d47f45a5962cfa3109df8872" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } } }
I20260807 05:13:44.883131 32079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32c3f22839954ec4bb46f17331761b4d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:13:44.888970 32126 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43619 } } peers { permanent_uuid: "32c3f22839954ec4bb46f17331761b4d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } } peers { permanent_uuid: "156f0048d47f45a5962cfa3109df8872" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } } }
I20260807 05:13:44.889109 32126 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:44.890334 32079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32c3f22839954ec4bb46f17331761b4d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43619 } } peers { permanent_uuid: "32c3f22839954ec4bb46f17331761b4d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } } peers { permanent_uuid: "156f0048d47f45a5962cfa3109df8872" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } } }
I20260807 05:13:44.890448 32079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32c3f22839954ec4bb46f17331761b4d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:13:44.890792 32132 mvcc.cc:205] Tried to move back new op lower bound from 7315782143459442688 to 7315782143349518336. Current Snapshot: MvccSnapshot[applied={T|T < 7315782143459442688}]
I20260807 05:13:44.892828 32135 mvcc.cc:205] Tried to move back new op lower bound from 7315782143459442688 to 7315782143349518336. Current Snapshot: MvccSnapshot[applied={T|T < 7315782143459442688}]
I20260807 05:13:44.893569 32078 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43619 } } peers { permanent_uuid: "32c3f22839954ec4bb46f17331761b4d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } } peers { permanent_uuid: "156f0048d47f45a5962cfa3109df8872" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } } }
I20260807 05:13:44.893683 32078 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:44.894708 32105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 156f0048d47f45a5962cfa3109df8872 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6c9ea3ac8d14212b0d784fdd4142f8b" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 43619 } } peers { permanent_uuid: "32c3f22839954ec4bb46f17331761b4d" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 34575 } } peers { permanent_uuid: "156f0048d47f45a5962cfa3109df8872" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 41529 } } }
I20260807 05:13:44.894820 32105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 156f0048d47f45a5962cfa3109df8872 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:13:44.895903 32133 mvcc.cc:205] Tried to move back new op lower bound from 7315782143459442688 to 7315782143349518336. Current Snapshot: MvccSnapshot[applied={T|T < 7315782143459442688}]
I20260807 05:13:44.901043 32129 catalog_manager.cc:1347] Generated new cluster ID: 38c57917a8ff485c9f03281ef0b2a701
I20260807 05:13:44.901114 32129 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:44.952560 32129 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:44.953987 32129 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:44.979003 32129 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6c9ea3ac8d14212b0d784fdd4142f8b: Generated new TSK 0
I20260807 05:13:44.979914 32129 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 05:13:45.014403 31994 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:45.011294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39890 (local address 127.8.16.125:34575)
0807 05:13:45.011436 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:45.011441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:45.013183 (+ 1742us) server_negotiation.cc:407] Connection header received
0807 05:13:45.013400 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:45.013404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:45.013486 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:45.013547 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:45.014260 (+ 713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39890: BlockingRecv error: recv got EOF from 127.0.0.1:39890 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 05:13:45.015882 31912 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:45.012244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54318 (local address 127.8.16.126:43619)
0807 05:13:45.012363 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:45.012368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:45.014956 (+ 2588us) server_negotiation.cc:407] Connection header received
0807 05:13:45.015144 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:45.015148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:45.015199 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:45.015250 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:45.015761 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54318: BlockingRecv error: recv got EOF from 127.0.0.1:54318 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260807 05:13:45.017253 32073 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:45.012893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50554 (local address 127.8.16.124:41529)
0807 05:13:45.013010 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:45.013015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:45.016323 (+ 3308us) server_negotiation.cc:407] Connection header received
0807 05:13:45.016568 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:45.016572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:45.016637 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:45.016683 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:45.017139 (+ 456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50554: BlockingRecv error: recv got EOF from 127.0.0.1:50554 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 05:13:45.021611 31994 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:45.020052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39902 (local address 127.8.16.125:34575)
0807 05:13:45.020156 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:45.020160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:45.020658 (+ 498us) server_negotiation.cc:407] Connection header received
0807 05:13:45.020853 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:45.020856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:45.020926 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:45.020974 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:45.021485 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39902: BlockingRecv error: recv got EOF from 127.0.0.1:39902 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260807 05:13:45.026834 31994 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:45.024991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39910 (local address 127.8.16.125:34575)
0807 05:13:45.025098 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:45.025102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:45.025821 (+ 719us) server_negotiation.cc:407] Connection header received
0807 05:13:45.026055 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:45.026059 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:45.026146 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:45.026195 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:45.026735 (+ 540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39910: BlockingRecv error: recv got EOF from 127.0.0.1:39910 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260807 05:13:45.032373 31994 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:45.030822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39920 (local address 127.8.16.125:34575)
0807 05:13:45.030932 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:45.030936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:45.031445 (+ 509us) server_negotiation.cc:407] Connection header received
0807 05:13:45.031634 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:45.031637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:45.031686 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:45.031735 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:45.032271 (+ 536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39920: BlockingRecv error: recv got EOF from 127.0.0.1:39920 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 05:13:45.040112 31994 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:45.038568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39928 (local address 127.8.16.125:34575)
0807 05:13:45.038712 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:45.038716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:45.039223 (+ 507us) server_negotiation.cc:407] Connection header received
0807 05:13:45.039399 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:45.039402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:45.039451 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:45.039498 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:45.039997 (+ 499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39928: BlockingRecv error: recv got EOF from 127.0.0.1:39928 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260807 05:13:45.051558 31994 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:45.050213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39932 (local address 127.8.16.125:34575)
0807 05:13:45.050331 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:45.050335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:45.050565 (+ 230us) server_negotiation.cc:407] Connection header received
0807 05:13:45.050792 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:45.050795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:45.050849 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:45.050899 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:45.051463 (+ 564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39932: BlockingRecv error: recv got EOF from 127.0.0.1:39932 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260807 05:13:45.071045 31994 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:45.069986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39942 (local address 127.8.16.125:34575)
0807 05:13:45.070137 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:45.070141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:45.070239 (+ 98us) server_negotiation.cc:407] Connection header received
0807 05:13:45.070363 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:45.070367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:45.070417 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:45.070507 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:45.070949 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39942: BlockingRecv error: recv got EOF from 127.0.0.1:39942 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 05:13:45.106405 31994 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:45.105213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39948 (local address 127.8.16.125:34575)
0807 05:13:45.105390 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:45.105396 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:45.105536 (+ 140us) server_negotiation.cc:407] Connection header received
0807 05:13:45.105716 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:45.105719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:45.105773 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:45.105849 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:45.106281 (+ 432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39948: BlockingRecv error: recv got EOF from 127.0.0.1:39948 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260807 05:13:45.173664 31994 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:45.172522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39950 (local address 127.8.16.125:34575)
0807 05:13:45.172660 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:45.172665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:45.172868 (+ 203us) server_negotiation.cc:407] Connection header received
0807 05:13:45.172985 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:45.172987 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:45.173035 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:45.173125 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:45.173542 (+ 417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39950: BlockingRecv error: recv got EOF from 127.0.0.1:39950 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 05:13:45.304811 31994 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:45.303661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39964 (local address 127.8.16.125:34575)
0807 05:13:45.303811 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:45.303816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:45.303957 (+ 141us) server_negotiation.cc:407] Connection header received
0807 05:13:45.304105 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:45.304108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:45.304155 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:45.304226 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:45.304683 (+ 457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39964: BlockingRecv error: recv got EOF from 127.0.0.1:39964 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 05:13:45.564095 31994 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:45.562906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39978 (local address 127.8.16.125:34575)
0807 05:13:45.563034 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:45.563040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:45.563217 (+ 177us) server_negotiation.cc:407] Connection header received
0807 05:13:45.563376 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:45.563379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:45.563427 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:45.563497 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:45.563982 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39978: BlockingRecv error: recv got EOF from 127.0.0.1:39978 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 05:13:45.837023 32103 catalog_manager.cc:1259] Loaded cluster ID: 38c57917a8ff485c9f03281ef0b2a701
I20260807 05:13:45.837118 32103 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 32c3f22839954ec4bb46f17331761b4d: loading cluster ID for follower catalog manager: success
I20260807 05:13:45.838830 32103 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 32c3f22839954ec4bb46f17331761b4d: acquiring CA information for follower catalog manager: success
I20260807 05:13:45.839969 32103 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 32c3f22839954ec4bb46f17331761b4d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 05:13:45.853696 32124 catalog_manager.cc:1259] Loaded cluster ID: 38c57917a8ff485c9f03281ef0b2a701
I20260807 05:13:45.853785 32124 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 156f0048d47f45a5962cfa3109df8872: loading cluster ID for follower catalog manager: success
I20260807 05:13:45.855368 32124 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 156f0048d47f45a5962cfa3109df8872: acquiring CA information for follower catalog manager: success
I20260807 05:13:45.856463 32124 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 156f0048d47f45a5962cfa3109df8872: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 05:13:46.081887 32210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:46.078177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46064 (local address 127.8.16.125:34575)
0807 05:13:46.078421 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:46.078425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:46.078506 (+ 81us) server_negotiation.cc:407] Connection header received
0807 05:13:46.078629 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:46.078631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:46.078769 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:46.078880 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:13:46.079088 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:46.079600 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:46.080463 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:46.080689 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:46.080776 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:46.081078 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:13:46.081541 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:13:46.081651 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:46.081695 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 05:13:46.081747 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":84,"threads_started":1}
I20260807 05:13:46.087877 32216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:46.084736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40466 (local address 127.8.16.124:41529)
0807 05:13:46.084942 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:46.084945 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:46.084957 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:13:46.085033 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:46.085035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:46.085140 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:46.085233 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:13:46.085364 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:46.085906 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:46.086701 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:46.086925 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:46.086998 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:46.087197 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:13:46.087570 (+ 373us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:13:46.087656 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:46.087707 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 05:13:46.087763 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":63,"threads_started":1}
I20260807 05:13:46.093972 32210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:46.090519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46074 (local address 127.8.16.125:34575)
0807 05:13:46.090651 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:46.090655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:46.090748 (+ 93us) server_negotiation.cc:407] Connection header received
0807 05:13:46.090876 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:46.090878 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:46.090921 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:46.090978 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:13:46.091184 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:46.091898 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:46.092822 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:46.093134 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:46.093223 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:46.093548 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:13:46.093621 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:13:46.093680 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:46.093782 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 05:13:46.093835 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 05:13:46.095149 32216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:46.091191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40478 (local address 127.8.16.124:41529)
0807 05:13:46.091794 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:46.091798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:46.091810 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:13:46.091847 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:46.091849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:46.091892 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:46.091953 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:13:46.092271 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:46.092984 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:46.094110 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:46.094410 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:46.094479 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:46.094761 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:13:46.094826 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:13:46.094883 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:46.094979 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 05:13:46.095028 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531}
I20260807 05:13:46.095535 32222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:46.090597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55364 (local address 127.8.16.126:43619)
0807 05:13:46.090820 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:46.090824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:46.091382 (+ 558us) server_negotiation.cc:407] Connection header received
0807 05:13:46.091512 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:46.091514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:46.091623 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:46.091726 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:13:46.091952 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:46.092640 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:46.093755 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:46.094092 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:46.094178 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:46.094573 (+ 395us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:13:46.094921 (+ 348us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:13:46.095004 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:46.095349 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 05:13:46.095413 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":66,"threads_started":1}
I20260807 05:13:46.098371 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 31841
I20260807 05:13:46.107177 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 31918
I20260807 05:13:46.113507 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 31996
2026-08-07T05:13:46Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3046 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:13:46 dist-test-slave-49h3 krb5kdc[32227](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:13:46 dist-test-slave-49h3 krb5kdc[32227](info): set up 2 sockets
Aug 07 05:13:46 dist-test-slave-49h3 krb5kdc[32227](info): commencing operation
krb5kdc: starting...
W20260807 05:13:47.093289 8257 mini_kdc.cc:121] Time spent starting KDC: real 0.959s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/test-user.keytab.
Aug 07 05:13:47 dist-test-slave-49h3 krb5kdc[32227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:13:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:13:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
Entry for principal kudu/127.8.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab.
I20260807 05:13:47.212546 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:36893
--webserver_interface=127.8.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:38007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:36893,127.8.16.125:35891,127.8.16.124:45081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:13:47.320492 32241 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 05:13:47.320765 32241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:47.320791 32241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:47.320848 32241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:47.324759 32241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:47.324815 32241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:47.324834 32241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:47.324862 32241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:47.324888 32241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:47.330112 32241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:38007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:36893,127.8.16.125:35891,127.8.16.124:45081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.126.keytab
--principal=kudu/127.8.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.126:36893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:47.331101 32241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:47.332080 32241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:47.340016 32250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:47.340061 32248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:47.340026 32247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:47.340279 32241 server_base.cc:1034] running on GCE node
I20260807 05:13:47.340816 32241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:47.341861 32241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:47.343010 32241 hybrid_clock.cc:648] HybridClock initialized: now 1786079627342997 us; error 25 us; skew 500 ppm
Aug 07 05:13:47 dist-test-slave-49h3 krb5kdc[32227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079627, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:47.348541 32241 init.cc:377] Logged in from keytab as kudu/127.8.16.126@KRBTEST.COM (short username kudu)
I20260807 05:13:47.349368 32241 webserver.cc:460] Webserver started at http://127.8.16.126:44603/ using document root <none> and password file <none>
I20260807 05:13:47.349903 32241 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:47.349964 32241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:47.350133 32241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:47.352183 32241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-0/data/instance:
uuid: "24d28c4a7ed5489eb4065363c4218411"
format_stamp: "Formatted at 2026-08-07 05:13:47 on dist-test-slave-49h3"
server_key: "4713dbee444d27b150f603674ef4c037"
server_key_iv: "ed76538baee50ef58a2dd2104a681800"
server_key_version: "encryptionkey@0"
I20260807 05:13:47.352700 32241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-0/wal/instance:
uuid: "24d28c4a7ed5489eb4065363c4218411"
format_stamp: "Formatted at 2026-08-07 05:13:47 on dist-test-slave-49h3"
server_key: "4713dbee444d27b150f603674ef4c037"
server_key_iv: "ed76538baee50ef58a2dd2104a681800"
server_key_version: "encryptionkey@0"
I20260807 05:13:47.356178 32241 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:13:47.358431 32257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:47.359567 32241 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:13:47.359673 32241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-0/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-0/wal
uuid: "24d28c4a7ed5489eb4065363c4218411"
format_stamp: "Formatted at 2026-08-07 05:13:47 on dist-test-slave-49h3"
server_key: "4713dbee444d27b150f603674ef4c037"
server_key_iv: "ed76538baee50ef58a2dd2104a681800"
server_key_version: "encryptionkey@0"
I20260807 05:13:47.359768 32241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:47.407675 32241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:47.410929 32241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:47.411118 32241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:47.418175 32241 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.126:36893
I20260807 05:13:47.418442 32311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.126:36893 every 8 connection(s)
I20260807 05:13:47.419124 32241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-0/data/info.pb
I20260807 05:13:47.419377 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 32241
I20260807 05:13:47.419492 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-0/wal/instance
I20260807 05:13:47.419817 8257 external_mini_cluster.cc:1428] Setting key 6d39f1c46e670d9b7adc294d64deea1d
I20260807 05:13:47.421906 32312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:47.425554 32312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36893 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.16.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.125@KRBTEST.COM" created.
Aug 07 05:13:47 dist-test-slave-49h3 krb5kdc[32227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079627, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:13:47.440711 32316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.428708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48592 (local address 127.8.16.126:36893)
0807 05:13:47.428967 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:47.428977 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:47.429012 (+ 35us) server_negotiation.cc:407] Connection header received
0807 05:13:47.429548 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:47.429553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:47.429780 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:47.429966 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:47.430790 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.431968 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.432935 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.433521 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.433698 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.435972 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:47.435980 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:47.435993 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:47.436020 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:47.437802 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.438225 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.438230 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.438231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.438338 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.438605 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.438607 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.438608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.439021 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:47.439164 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:47.439520 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 05:13:47.439785 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":114,"server-negotiator.queue_time_us":173,"thread_start_us":76,"threads_started":1}
I20260807 05:13:47.440711 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.427902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36893 (local address 127.0.0.1:48592)
0807 05:13:47.428417 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.428459 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:47.429261 (+ 802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:47.429962 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:47.429967 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:47.430410 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:47.430649 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:47.430662 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.432103 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.432106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:47.432823 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:47.432832 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.433659 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.433662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:47.433739 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.434395 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:47.434411 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:47.435820 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:47.437903 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:47.437910 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:47.437918 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:47.438132 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:47.438414 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:47.438416 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:47.438417 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:47.438528 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:47.439157 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:47.439162 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:47.439294 (+ 132us) client_negotiation.cc:769] Sending connection context
0807 05:13:47.439427 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 05:13:47.439699 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":2,"spinlock_wait_cycles":12416,"thread_start_us":143,"threads_started":1}
I20260807 05:13:47.442986 32312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:13:47.443632 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.443392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:35891 (local address 127.0.0.1:41746)
0807 05:13:47.443504 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.443570 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.125:35891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 05:13:47.443799 32263 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.16.125:35891: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.125.keytab.
Entry for principal kudu/127.8.16.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.125.keytab.
I20260807 05:13:47.444610 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.444383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:35891 (local address 127.0.0.1:41754)
0807 05:13:47.444490 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.444542 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.125:35891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 05:13:47.444808 32312 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.125:35891: Network error: Client connection negotiation failed: client connection to 127.8.16.125:35891: connect: Connection refused (error 111)
I20260807 05:13:47.445267 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.125:35891
--webserver_interface=127.8.16.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.125.keytab
--principal=kudu/127.8.16.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:38007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:36893,127.8.16.125:35891,127.8.16.124:45081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:13:47.494063 32312 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } attempt: 1
I20260807 05:13:47.494951 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.494650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:35891 (local address 127.0.0.1:41770)
0807 05:13:47.494807 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.494870 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.125:35891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 05:13:47.495636 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.495432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:35891 (local address 127.0.0.1:41786)
0807 05:13:47.495533 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.495578 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.125:35891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 05:13:47.495760 32312 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.125:35891: Network error: Client connection negotiation failed: client connection to 127.8.16.125:35891: connect: Connection refused (error 111)
W20260807 05:13:47.554011 32319 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 05:13:47.554306 32319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:47.554349 32319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:47.554414 32319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:47.558394 32319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:47.558452 32319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:47.558478 32319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:47.558501 32319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:47.558529 32319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:47.563745 32319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:38007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:36893,127.8.16.125:35891,127.8.16.124:45081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.125.keytab
--principal=kudu/127.8.16.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.125:35891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.16.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:47.563900 32312 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } attempt: 2
I20260807 05:13:47.564644 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.564364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:35891 (local address 127.0.0.1:41792)
0807 05:13:47.564509 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.564569 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.125:35891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 05:13:47.564822 32319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:47.565376 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.565179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:35891 (local address 127.0.0.1:41798)
0807 05:13:47.565277 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.565320 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.125:35891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 05:13:47.565513 32312 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.125:35891: Network error: Client connection negotiation failed: client connection to 127.8.16.125:35891: connect: Connection refused (error 111)
I20260807 05:13:47.565990 32319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:13:47.574028 32325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:47.574046 32324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:47.574039 32327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:47.574218 32319 server_base.cc:1034] running on GCE node
I20260807 05:13:47.574844 32319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:47.575868 32319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:47.577014 32319 hybrid_clock.cc:648] HybridClock initialized: now 1786079627576988 us; error 38 us; skew 500 ppm
Aug 07 05:13:47 dist-test-slave-49h3 krb5kdc[32227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079627, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:47.579605 32319 init.cc:377] Logged in from keytab as kudu/127.8.16.125@KRBTEST.COM (short username kudu)
I20260807 05:13:47.580461 32319 webserver.cc:460] Webserver started at http://127.8.16.125:46665/ using document root <none> and password file <none>
I20260807 05:13:47.581003 32319 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:47.581071 32319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:47.581260 32319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:47.583273 32319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-1/data/instance:
uuid: "7adff94de91146b59844dfa270c8eed2"
format_stamp: "Formatted at 2026-08-07 05:13:47 on dist-test-slave-49h3"
server_key: "23f83d074eacb9b8b11913e88e4a5bfb"
server_key_iv: "5c6c8bec156dc30b6a97489dbd541f54"
server_key_version: "encryptionkey@0"
I20260807 05:13:47.583792 32319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-1/wal/instance:
uuid: "7adff94de91146b59844dfa270c8eed2"
format_stamp: "Formatted at 2026-08-07 05:13:47 on dist-test-slave-49h3"
server_key: "23f83d074eacb9b8b11913e88e4a5bfb"
server_key_iv: "5c6c8bec156dc30b6a97489dbd541f54"
server_key_version: "encryptionkey@0"
I20260807 05:13:47.587396 32319 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260807 05:13:47.589593 32334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:47.590720 32319 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:13:47.590838 32319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-1/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-1/wal
uuid: "7adff94de91146b59844dfa270c8eed2"
format_stamp: "Formatted at 2026-08-07 05:13:47 on dist-test-slave-49h3"
server_key: "23f83d074eacb9b8b11913e88e4a5bfb"
server_key_iv: "5c6c8bec156dc30b6a97489dbd541f54"
server_key_version: "encryptionkey@0"
I20260807 05:13:47.590927 32319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:47.616382 32319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:47.619261 32319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:47.619453 32319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:47.626267 32319 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.125:35891
I20260807 05:13:47.626520 32389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.125:35891 every 8 connection(s)
I20260807 05:13:47.627246 32319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-1/data/info.pb
I20260807 05:13:47.629895 32390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:47.631469 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 32319
I20260807 05:13:47.631619 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-1/wal/instance
I20260807 05:13:47.631940 8257 external_mini_cluster.cc:1428] Setting key 09d2172d648693929b3339c2a46071d1
I20260807 05:13:47.634325 32390 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36893 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.16.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.16.124@KRBTEST.COM" created.
Aug 07 05:13:47 dist-test-slave-49h3 krb5kdc[32227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079627, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.125@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:13:47.647764 32316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.636171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48594 (local address 127.8.16.126:36893)
0807 05:13:47.636301 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:47.636305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:47.637385 (+ 1080us) server_negotiation.cc:407] Connection header received
0807 05:13:47.638312 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:47.638315 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:47.638378 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:47.638449 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:47.639568 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.640089 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.641593 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.641825 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.641915 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.644378 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:47.644381 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:47.644384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:47.644413 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:47.645771 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.646279 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.646282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.646284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.646373 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.646651 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.646654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.646655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.646872 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:47.646988 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:47.647465 (+ 477us) server_negotiation.cc:299] Negotiation successful
0807 05:13:47.647595 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 05:13:47.648469 32393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.636738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36893 (local address 127.0.0.1:48594)
0807 05:13:47.637231 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.637273 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:47.638109 (+ 836us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:47.638615 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:47.638622 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:47.638972 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:47.639425 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:47.639438 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.640208 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.640211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:47.641472 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:47.641485 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.641962 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.641967 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:47.642125 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.642649 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:47.642665 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:47.644206 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:47.645881 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:47.645886 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:47.645900 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:47.646184 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:47.646451 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:47.646453 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:47.646454 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:47.646575 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:47.646965 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:47.646970 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:47.647223 (+ 253us) client_negotiation.cc:769] Sending connection context
0807 05:13:47.647376 (+ 153us) client_negotiation.cc:240] Negotiation successful
0807 05:13:47.647624 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":139,"threads_started":1}
I20260807 05:13:47.649904 32390 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 05:13:47 dist-test-slave-49h3 krb5kdc[32227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079627, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.125@KRBTEST.COM for kudu/127.8.16.125@KRBTEST.COM
I20260807 05:13:47.656658 32312 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.16.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.124.keytab.
Entry for principal kudu/127.8.16.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.124.keytab.
I20260807 05:13:47.658710 8257 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
/tmp/dist-test-taskg_K4zd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.124:45081
--webserver_interface=127.8.16.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.124.keytab
--principal=kudu/127.8.16.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.16.84:38007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.16.126:36893,127.8.16.125:35891,127.8.16.124:45081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:13:47.658921 32393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.650396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:35891 (local address 127.0.0.1:41808)
0807 05:13:47.650551 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.650562 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:47.650632 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:47.651147 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:47.651150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:47.651307 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:47.651388 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:47.651392 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.652504 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.652507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:47.653049 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:47.653057 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.653734 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.653737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:47.653833 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.654291 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:47.654302 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:47.655446 (+ 1144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:47.657298 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:47.657301 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:47.657303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:47.657552 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:47.657866 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:47.657868 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:47.657870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:47.657949 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:47.658482 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:47.658484 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:47.658560 (+ 76us) client_negotiation.cc:769] Sending connection context
0807 05:13:47.658647 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 05:13:47.658770 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260807 05:13:47.659051 32395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.650512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41808 (local address 127.8.16.125:35891)
0807 05:13:47.650831 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:47.650840 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:47.650856 (+ 16us) server_negotiation.cc:407] Connection header received
0807 05:13:47.650886 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:47.650890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:47.651028 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:47.651172 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:47.651482 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.652407 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.653149 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.653613 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.653764 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.655589 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:47.655599 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:47.655609 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:47.655630 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:47.657130 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.657651 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.657657 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.657658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.657766 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.658061 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.658065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.658066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.658384 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:47.658522 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:47.658756 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 05:13:47.658880 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":144,"threads_started":1}
I20260807 05:13:47.661078 32390 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:13:47.661778 32393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.661533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:45081 (local address 127.0.0.1:60230)
0807 05:13:47.661646 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.661718 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:45081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 05:13:47.661988 32337 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.16.124:45081: connect: Connection refused (error 111)
Aug 07 05:13:47 dist-test-slave-49h3 krb5kdc[32227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079627, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for kudu/127.8.16.125@KRBTEST.COM
I20260807 05:13:47.663026 32393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.662812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:45081 (local address 127.0.0.1:60238)
0807 05:13:47.662913 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.662963 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:45081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 05:13:47.663482 32390 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.124:45081: Network error: Client connection negotiation failed: client connection to 127.8.16.124:45081: connect: Connection refused (error 111)
I20260807 05:13:47.666369 32396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.657138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41818 (local address 127.8.16.125:35891)
0807 05:13:47.657407 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:47.657415 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:47.657427 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:13:47.657481 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:47.657484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:47.657525 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:47.657611 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:47.658567 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.659721 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.660950 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.661187 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.661321 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.663437 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:47.663440 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:47.663447 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:47.663472 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:47.664852 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.665303 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.665305 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.665306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.665371 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.665646 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.665648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.665649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.665806 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:47.665878 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:47.666165 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 05:13:47.666253 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":103,"threads_started":1}
I20260807 05:13:47.666428 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.657110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:35891 (local address 127.0.0.1:41818)
0807 05:13:47.657243 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.657259 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:47.657338 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:47.657631 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:47.657634 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:47.657814 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:47.657938 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:47.657946 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.659838 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.659841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:47.660799 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:47.660812 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.661338 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.661342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:47.661490 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.661941 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:47.661954 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:47.663307 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:47.664958 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:47.664962 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:47.664964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:47.665208 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:47.665466 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:47.665469 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:47.665471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:47.665565 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:47.665889 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:47.665892 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:47.666000 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 05:13:47.666122 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 05:13:47.666271 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 05:13:47.667075 32312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:13:47.667723 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.667468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:45081 (local address 127.0.0.1:60244)
0807 05:13:47.667590 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.667649 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:45081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 05:13:47.668412 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.668194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:45081 (local address 127.0.0.1:60256)
0807 05:13:47.668323 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.668364 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:45081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260807 05:13:47.668529 32312 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.124:45081: Network error: Client connection negotiation failed: client connection to 127.8.16.124:45081: connect: Connection refused (error 111)
I20260807 05:13:47.699671 32312 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } attempt: 1
I20260807 05:13:47.700541 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.700244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:45081 (local address 127.0.0.1:60258)
0807 05:13:47.700394 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.700456 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:45081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 05:13:47.701213 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.701018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:45081 (local address 127.0.0.1:60274)
0807 05:13:47.701102 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.701145 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:45081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 05:13:47.701333 32312 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.124:45081: Network error: Client connection negotiation failed: client connection to 127.8.16.124:45081: connect: Connection refused (error 111)
I20260807 05:13:47.712756 32390 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } attempt: 1
I20260807 05:13:47.713583 32393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.713280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:45081 (local address 127.0.0.1:60288)
0807 05:13:47.713413 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.713500 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:45081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 05:13:47.714247 32393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.714029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:45081 (local address 127.0.0.1:60294)
0807 05:13:47.714127 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.714188 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:45081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 05:13:47.714388 32390 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.124:45081: Network error: Client connection negotiation failed: client connection to 127.8.16.124:45081: connect: Connection refused (error 111)
W20260807 05:13:47.770877 32397 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 05:13:47.771162 32397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:13:47.771204 32397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:13:47.771267 32397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:13:47.775297 32397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:13:47.775364 32397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:13:47.775389 32397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:13:47.775411 32397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:13:47.775440 32397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:13:47.776505 32312 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } attempt: 2
I20260807 05:13:47.777393 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.777056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:45081 (local address 127.0.0.1:60302)
0807 05:13:47.777236 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.777309 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:45081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 05:13:47.778095 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.777885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:45081 (local address 127.0.0.1:60318)
0807 05:13:47.777989 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.778034 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:45081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 05:13:47.778218 32312 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.124:45081: Network error: Client connection negotiation failed: client connection to 127.8.16.124:45081: connect: Connection refused (error 111)
I20260807 05:13:47.781044 32397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.16.84:38007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.16.126:36893,127.8.16.125:35891,127.8.16.124:45081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/krb5kdc/kudu_127.8.16.124.keytab
--principal=kudu/127.8.16.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.16.124:45081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.16.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:13:47.782176 32397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:13:47.782523 32390 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } attempt: 2
I20260807 05:13:47.783222 32393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.782967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:45081 (local address 127.0.0.1:60320)
0807 05:13:47.783095 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.783149 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:45081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 05:13:47.783325 32397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 05:13:47.783854 32393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.783655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:45081 (local address 127.0.0.1:60336)
0807 05:13:47.783757 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.783800 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.16.124:45081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 05:13:47.783993 32390 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.16.124:45081: Network error: Client connection negotiation failed: client connection to 127.8.16.124:45081: connect: Connection refused (error 111)
W20260807 05:13:47.791452 32403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:47.791452 32404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:13:47.791489 32406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:13:47.791720 32397 server_base.cc:1034] running on GCE node
I20260807 05:13:47.792330 32397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:13:47.793427 32397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:13:47.794621 32397 hybrid_clock.cc:648] HybridClock initialized: now 1786079627794588 us; error 46 us; skew 500 ppm
Aug 07 05:13:47 dist-test-slave-49h3 krb5kdc[32227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079627, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:13:47.797313 32397 init.cc:377] Logged in from keytab as kudu/127.8.16.124@KRBTEST.COM (short username kudu)
I20260807 05:13:47.798252 32397 webserver.cc:460] Webserver started at http://127.8.16.124:40349/ using document root <none> and password file <none>
I20260807 05:13:47.798796 32397 fs_manager.cc:359] Metadata directory not provided
I20260807 05:13:47.798864 32397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:13:47.799068 32397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:13:47.801126 32397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-2/data/instance:
uuid: "b78e673c552a475da13283e4193637c7"
format_stamp: "Formatted at 2026-08-07 05:13:47 on dist-test-slave-49h3"
server_key: "19e6da488073c69bf2bf932b1ab0f709"
server_key_iv: "de6e1f7c638343bb572084bb67bbc906"
server_key_version: "encryptionkey@0"
I20260807 05:13:47.801715 32397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-2/wal/instance:
uuid: "b78e673c552a475da13283e4193637c7"
format_stamp: "Formatted at 2026-08-07 05:13:47 on dist-test-slave-49h3"
server_key: "19e6da488073c69bf2bf932b1ab0f709"
server_key_iv: "de6e1f7c638343bb572084bb67bbc906"
server_key_version: "encryptionkey@0"
I20260807 05:13:47.805397 32397 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 05:13:47.807780 32413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:13:47.808902 32397 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:13:47.809010 32397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-2/data,/tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-2/wal
uuid: "b78e673c552a475da13283e4193637c7"
format_stamp: "Formatted at 2026-08-07 05:13:47 on dist-test-slave-49h3"
server_key: "19e6da488073c69bf2bf932b1ab0f709"
server_key_iv: "de6e1f7c638343bb572084bb67bbc906"
server_key_version: "encryptionkey@0"
I20260807 05:13:47.809113 32397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:13:47.830823 32397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:13:47.834259 32397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:13:47.834460 32397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:13:47.841274 32397 rpc_server.cc:307] RPC server started. Bound to: 127.8.16.124:45081
I20260807 05:13:47.841570 32467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.16.124:45081 every 8 connection(s)
I20260807 05:13:47.842360 32397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-2/data/info.pb
I20260807 05:13:47.845220 32468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:13:47.846632 8257 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu as pid 32397
I20260807 05:13:47.846791 8257 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskg_K4zd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079354071073-8257-0/minicluster-data/master-2/wal/instance
I20260807 05:13:47.847085 8257 external_mini_cluster.cc:1428] Setting key 33ccf062aa59ecb1d895b901309add23
I20260807 05:13:47.849617 32468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36893 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 05:13:47 dist-test-slave-49h3 krb5kdc[32227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.124@KRBTEST.COM
Aug 07 05:13:47 dist-test-slave-49h3 krb5kdc[32227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079627, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.124@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:13:47.860760 32316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.851718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48602 (local address 127.8.16.126:36893)
0807 05:13:47.851862 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:47.851867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:47.852280 (+ 413us) server_negotiation.cc:407] Connection header received
0807 05:13:47.852402 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:47.852404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:47.852446 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:47.852511 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:47.853245 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.853987 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.855025 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.855237 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.855321 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.857585 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:47.857589 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:47.857592 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:47.857621 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:47.858922 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.859394 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.859397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.859399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.859484 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.859780 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.859783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.859784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.860000 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:47.860101 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:47.860466 (+ 365us) server_negotiation.cc:299] Negotiation successful
0807 05:13:47.860591 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 05:13:47.861629 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.848474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60338 (local address 127.8.16.124:45081)
0807 05:13:47.848904 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:47.848922 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:47.848966 (+ 44us) server_negotiation.cc:407] Connection header received
0807 05:13:47.849820 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:47.849844 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:47.850305 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:47.850667 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:47.851341 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.852812 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.853899 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.854531 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.854681 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.857107 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:47.857114 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:47.857127 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:47.857149 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:47.858779 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.859236 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.859241 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.859243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.859351 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.859610 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.859613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.859614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.860077 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:47.860259 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:47.860463 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 05:13:47.860710 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":125,"threads_started":1}
I20260807 05:13:47.861627 32472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.851627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.126:36893 (local address 127.0.0.1:48602)
0807 05:13:47.852147 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.852199 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:47.852321 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:47.852567 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:47.852571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:47.853005 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:47.853123 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:47.853129 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.854086 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.854088 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:47.854915 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:47.854922 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.855370 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.855374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:47.855470 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.855940 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:47.855955 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:47.857426 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:47.859011 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:47.859019 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:47.859029 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:47.859308 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:47.859562 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:47.859566 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:47.859568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:47.859693 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:47.860088 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:47.860093 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:47.860252 (+ 159us) client_negotiation.cc:769] Sending connection context
0807 05:13:47.860420 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 05:13:47.860825 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":102,"thread_start_us":151,"threads_started":1}
I20260807 05:13:47.862789 32468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 05:13:47 dist-test-slave-49h3 krb5kdc[32227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079627, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.124@KRBTEST.COM for kudu/127.8.16.125@KRBTEST.COM
Aug 07 05:13:47 dist-test-slave-49h3 krb5kdc[32227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.126@KRBTEST.COM
I20260807 05:13:47.871452 32396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.863379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41826 (local address 127.8.16.125:35891)
0807 05:13:47.863490 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:47.863494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:47.863509 (+ 15us) server_negotiation.cc:407] Connection header received
0807 05:13:47.863612 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:47.863614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:47.863666 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:47.863751 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:47.864328 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.865093 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.866092 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.866470 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.866571 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.868475 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:47.868480 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:47.868483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:47.868513 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:47.869926 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.870330 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.870332 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.870334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.870423 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.870666 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.870669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.870671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.870894 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:47.870995 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:47.871223 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 05:13:47.871329 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 05:13:47.871464 32472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.863285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.125:35891 (local address 127.0.0.1:41826)
0807 05:13:47.863439 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.863454 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:47.863527 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:47.863770 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:47.863773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:47.863945 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:47.864200 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:47.864209 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.865196 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.865199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:47.865985 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:47.865993 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.866563 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.866565 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:47.866634 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.867058 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:47.867073 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:47.868318 (+ 1245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:47.870038 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:47.870041 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:47.870043 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:47.870236 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:47.870511 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:47.870513 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:47.870514 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:47.870589 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:47.870981 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:47.870984 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:47.871061 (+ 77us) client_negotiation.cc:769] Sending connection context
0807 05:13:47.871152 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 05:13:47.871295 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 05:13:47.872273 32468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:13:47.872912 32316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.863990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48614 (local address 127.8.16.126:36893)
0807 05:13:47.864103 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:47.864107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:47.864190 (+ 83us) server_negotiation.cc:407] Connection header received
0807 05:13:47.864280 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:47.864282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:47.864327 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:47.864381 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:47.865064 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.865803 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.866754 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.867005 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.867104 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.869598 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:47.869602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:47.869604 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:47.869631 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:47.871064 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.871486 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.871489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.871491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.871575 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.871881 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.871885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.871886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.872093 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:47.872291 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:47.872642 (+ 351us) server_negotiation.cc:299] Negotiation successful
0807 05:13:47.872773 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 05:13:47.875136 32390 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } attempt: 3
Aug 07 05:13:47 dist-test-slave-49h3 krb5kdc[32227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079627, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.124@KRBTEST.COM for kudu/127.8.16.124@KRBTEST.COM
I20260807 05:13:47.877374 32312 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } attempt: 3
Aug 07 05:13:47 dist-test-slave-49h3 krb5kdc[32227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.16.125@KRBTEST.COM
I20260807 05:13:47.880383 32472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.872769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:45081 (local address 127.0.0.1:60342)
0807 05:13:47.872920 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.872933 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:47.873002 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:47.873235 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:47.873238 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:47.873359 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:47.873440 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:47.873478 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.874394 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.874397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:47.874879 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:47.874885 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.875265 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.875269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:47.875335 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.875761 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:47.875771 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:47.877033 (+ 1262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:47.878970 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:47.878973 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:47.878975 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:47.879191 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:47.879480 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:47.879483 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:47.879485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:47.879568 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:47.879958 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:47.879961 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:47.880029 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 05:13:47.880098 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 05:13:47.880209 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
Aug 07 05:13:47 dist-test-slave-49h3 krb5kdc[32227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079627, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.125@KRBTEST.COM for kudu/127.8.16.124@KRBTEST.COM
I20260807 05:13:47.880939 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.872845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60342 (local address 127.8.16.124:45081)
0807 05:13:47.872944 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:47.872948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:47.872964 (+ 16us) server_negotiation.cc:407] Connection header received
0807 05:13:47.873084 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:47.873087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:47.873125 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:47.873203 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:47.873774 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.874304 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.874972 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.875179 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.875266 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.877137 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:47.877140 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:47.877143 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:47.877163 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:47.878869 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.879303 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.879305 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.879307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.879381 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.879667 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.879670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.879672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.879866 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:47.880632 (+ 766us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:47.880696 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 05:13:47.880801 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Aug 07 05:13:47 dist-test-slave-49h3 krb5kdc[32227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079627, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.16.126@KRBTEST.COM for kudu/127.8.16.124@KRBTEST.COM
I20260807 05:13:47.884370 32393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.875466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:45081 (local address 127.0.0.1:60350)
0807 05:13:47.875613 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.875626 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:47.875691 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:47.876085 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:47.876087 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:47.876248 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:47.876316 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:47.876320 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.877618 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.877622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:47.878348 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:47.878359 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.879053 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.879057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:47.879137 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.879721 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:47.879735 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:47.881100 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:47.882864 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:47.882868 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:47.882870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:47.883100 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:47.883406 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:47.883409 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:47.883411 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:47.883493 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:47.883930 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:47.883932 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:47.884021 (+ 89us) client_negotiation.cc:769] Sending connection context
0807 05:13:47.884095 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 05:13:47.884222 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 05:13:47.884466 32468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b78e673c552a475da13283e4193637c7: Bootstrap starting.
I20260807 05:13:47.884737 32396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.874015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41830 (local address 127.8.16.125:35891)
0807 05:13:47.874110 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:47.874114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:47.874126 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:13:47.874234 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:47.874236 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:47.874272 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:47.874336 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:47.874888 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.875929 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.876955 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.877292 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.877971 (+ 679us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.880648 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:47.880653 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:47.880656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:47.880683 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:47.882590 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.883070 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.883073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.883075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.883164 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.883486 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.883489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.883490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.883688 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:47.884425 (+ 737us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:47.884501 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 05:13:47.884602 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 05:13:47.884970 32473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.875528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60350 (local address 127.8.16.124:45081)
0807 05:13:47.875911 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:47.875919 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:47.875931 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:13:47.875961 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:47.875963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:47.876003 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:47.876074 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:47.876417 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.877498 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.878494 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.878893 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.879323 (+ 430us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.881226 (+ 1903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:47.881230 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:47.881232 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:47.881255 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:47.882748 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.883220 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.883223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.883224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.883306 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.883596 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.883599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.883600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.883819 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:47.884632 (+ 813us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:47.884715 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 05:13:47.884818 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":91,"threads_started":1}
I20260807 05:13:47.887279 32468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b78e673c552a475da13283e4193637c7: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:47.888134 32468 log.cc:824] T 00000000000000000000000000000000 P b78e673c552a475da13283e4193637c7: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:47.888640 8257 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:13:47.888815 32390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7adff94de91146b59844dfa270c8eed2: Bootstrap starting.
I20260807 05:13:47.890373 32468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b78e673c552a475da13283e4193637c7: No bootstrap required, opened a new log
I20260807 05:13:47.891453 32390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7adff94de91146b59844dfa270c8eed2: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:47.892256 32390 log.cc:824] T 00000000000000000000000000000000 P 7adff94de91146b59844dfa270c8eed2: Log is configured to *not* fsync() on all Append() calls
I20260807 05:13:47.894284 32390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7adff94de91146b59844dfa270c8eed2: No bootstrap required, opened a new log
I20260807 05:13:47.894635 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.877784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.16.124:45081 (local address 127.0.0.1:60362)
0807 05:13:47.878290 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0807 05:13:47.878304 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:13:47.878377 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:13:47.879036 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:13:47.879039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:13:47.879203 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:13:47.879547 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:47.879556 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.880583 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.880586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:47.881946 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:13:47.881962 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.882765 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.882769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:13:47.882854 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.883263 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:13:47.883278 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:13:47.884573 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:13:47.892754 (+ 8181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:47.892758 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:47.892760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:47.893050 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:47.893399 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:13:47.893403 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:13:47.893404 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:13:47.893551 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:13:47.894077 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:13:47.894081 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:13:47.894206 (+ 125us) client_negotiation.cc:769] Sending connection context
0807 05:13:47.894320 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 05:13:47.894475 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424}
I20260807 05:13:47.895015 32474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.878549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60362 (local address 127.8.16.124:45081)
0807 05:13:47.878817 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:47.878825 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:47.878838 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:13:47.878868 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:47.878871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:47.878908 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:47.879633 (+ 725us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:47.879748 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.880461 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.882281 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.882634 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.884694 (+ 2060us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.884802 (+ 108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:47.884805 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:47.884807 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:47.884824 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:47.886410 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.893186 (+ 6776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.893189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.893191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.893293 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.893729 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.893732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.893734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.893960 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:47.894065 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:47.894454 (+ 389us) server_negotiation.cc:299] Negotiation successful
0807 05:13:47.894584 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":101,"threads_started":1}
I20260807 05:13:47.894870 32468 raft_consensus.cc:348] T 00000000000000000000000000000000 P b78e673c552a475da13283e4193637c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d28c4a7ed5489eb4065363c4218411" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36893 } } peers { permanent_uuid: "7adff94de91146b59844dfa270c8eed2" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } } peers { permanent_uuid: "b78e673c552a475da13283e4193637c7" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } }
I20260807 05:13:47.895107 32468 raft_consensus.cc:729] T 00000000000000000000000000000000 P b78e673c552a475da13283e4193637c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b78e673c552a475da13283e4193637c7, State: Initialized, Role: FOLLOWER
I20260807 05:13:47.895440 32468 consensus_queue.cc:260] T 00000000000000000000000000000000 P b78e673c552a475da13283e4193637c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d28c4a7ed5489eb4065363c4218411" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36893 } } peers { permanent_uuid: "7adff94de91146b59844dfa270c8eed2" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } } peers { permanent_uuid: "b78e673c552a475da13283e4193637c7" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } }
I20260807 05:13:47.896767 32468 sys_catalog.cc:564] T 00000000000000000000000000000000 P b78e673c552a475da13283e4193637c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:47.897403 32390 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7adff94de91146b59844dfa270c8eed2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d28c4a7ed5489eb4065363c4218411" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36893 } } peers { permanent_uuid: "7adff94de91146b59844dfa270c8eed2" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } } peers { permanent_uuid: "b78e673c552a475da13283e4193637c7" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } }
I20260807 05:13:47.897621 32390 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7adff94de91146b59844dfa270c8eed2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7adff94de91146b59844dfa270c8eed2, State: Initialized, Role: FOLLOWER
I20260807 05:13:47.897861 32477 sys_catalog.cc:455] T 00000000000000000000000000000000 P b78e673c552a475da13283e4193637c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d28c4a7ed5489eb4065363c4218411" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36893 } } peers { permanent_uuid: "7adff94de91146b59844dfa270c8eed2" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } } peers { permanent_uuid: "b78e673c552a475da13283e4193637c7" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } } }
I20260807 05:13:47.897943 32312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411: Bootstrap starting.
I20260807 05:13:47.897925 32390 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7adff94de91146b59844dfa270c8eed2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d28c4a7ed5489eb4065363c4218411" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36893 } } peers { permanent_uuid: "7adff94de91146b59844dfa270c8eed2" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } } peers { permanent_uuid: "b78e673c552a475da13283e4193637c7" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } }
I20260807 05:13:47.898089 32477 sys_catalog.cc:458] T 00000000000000000000000000000000 P b78e673c552a475da13283e4193637c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:13:47.897585 32286 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "24d28c4a7ed5489eb4065363c4218411"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48614
I20260807 05:13:47.899210 32390 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7adff94de91146b59844dfa270c8eed2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:47.900293 32478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7adff94de91146b59844dfa270c8eed2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d28c4a7ed5489eb4065363c4218411" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36893 } } peers { permanent_uuid: "7adff94de91146b59844dfa270c8eed2" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } } peers { permanent_uuid: "b78e673c552a475da13283e4193637c7" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } } }
I20260807 05:13:47.900475 32478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7adff94de91146b59844dfa270c8eed2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:13:47.901748 32312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411: Neither blocks nor log segments found. Creating new log.
I20260807 05:13:47.902410 32312 log.cc:824] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411: Log is configured to *not* fsync() on all Append() calls
W20260807 05:13:47.903260 32490 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b78e673c552a475da13283e4193637c7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:13:47.903337 32490 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:13:47.901602 32397 master_runner.cc:186] Error getting master registration for 127.8.16.126:36893: OK, instance_id { permanent_uuid: "24d28c4a7ed5489eb4065363c4218411" instance_seqno: 1786079627416207 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 05:13:47.904695 32397 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 05:13:47.904801 32312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411: No bootstrap required, opened a new log
I20260807 05:13:47.907804 32312 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d28c4a7ed5489eb4065363c4218411" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36893 } } peers { permanent_uuid: "7adff94de91146b59844dfa270c8eed2" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } } peers { permanent_uuid: "b78e673c552a475da13283e4193637c7" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } }
I20260807 05:13:47.908030 32312 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d28c4a7ed5489eb4065363c4218411, State: Initialized, Role: FOLLOWER
I20260807 05:13:47.908365 32312 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d28c4a7ed5489eb4065363c4218411" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36893 } } peers { permanent_uuid: "7adff94de91146b59844dfa270c8eed2" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } } peers { permanent_uuid: "b78e673c552a475da13283e4193637c7" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } }
I20260807 05:13:47.905531 32319 master_runner.cc:186] Error getting master registration for 127.8.16.126:36893: OK, instance_id { permanent_uuid: "24d28c4a7ed5489eb4065363c4218411" instance_seqno: 1786079627416207 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 05:13:47.908825 32319 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 05:13:47.909744 32312 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:13:47.909829 32286 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 05:13:47.909948 32286 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:47.911222 32286 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d28c4a7ed5489eb4065363c4218411" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36893 } } peers { permanent_uuid: "7adff94de91146b59844dfa270c8eed2" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } } peers { permanent_uuid: "b78e673c552a475da13283e4193637c7" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } }
W20260807 05:13:47.912147 32502 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7adff94de91146b59844dfa270c8eed2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:13:47.912266 32502 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:13:47.912248 32286 leader_election.cc:288] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411 [CANDIDATE]: Term 1 election: Requested vote from peers 7adff94de91146b59844dfa270c8eed2 (127.8.16.125:35891), b78e673c552a475da13283e4193637c7 (127.8.16.124:45081)
W20260807 05:13:47.912734 32265 tablet.cc:2369] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:13:47.913641 32441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "24d28c4a7ed5489eb4065363c4218411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b78e673c552a475da13283e4193637c7"
I20260807 05:13:47.913666 32362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "24d28c4a7ed5489eb4065363c4218411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7adff94de91146b59844dfa270c8eed2"
I20260807 05:13:47.913887 32441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b78e673c552a475da13283e4193637c7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:47.913887 32362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7adff94de91146b59844dfa270c8eed2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:13:47.915220 32441 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b78e673c552a475da13283e4193637c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d28c4a7ed5489eb4065363c4218411 in term 1.
I20260807 05:13:47.915592 32362 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7adff94de91146b59844dfa270c8eed2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d28c4a7ed5489eb4065363c4218411 in term 1.
I20260807 05:13:47.917610 32506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d28c4a7ed5489eb4065363c4218411" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36893 } } peers { permanent_uuid: "7adff94de91146b59844dfa270c8eed2" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } } peers { permanent_uuid: "b78e673c552a475da13283e4193637c7" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } } }
I20260807 05:13:47.917737 32506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:13:47.918534 32261 leader_election.cc:302] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24d28c4a7ed5489eb4065363c4218411, b78e673c552a475da13283e4193637c7; no voters:
I20260807 05:13:47.918843 32506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:13:47.919109 32506 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411 [term 1 LEADER]: Becoming Leader. State: Replica: 24d28c4a7ed5489eb4065363c4218411, State: Running, Role: LEADER
I20260807 05:13:47.919500 32506 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d28c4a7ed5489eb4065363c4218411" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36893 } } peers { permanent_uuid: "7adff94de91146b59844dfa270c8eed2" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } } peers { permanent_uuid: "b78e673c552a475da13283e4193637c7" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } }
I20260807 05:13:47.921304 32506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24d28c4a7ed5489eb4065363c4218411. Latest consensus state: current_term: 1 leader_uuid: "24d28c4a7ed5489eb4065363c4218411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d28c4a7ed5489eb4065363c4218411" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36893 } } peers { permanent_uuid: "7adff94de91146b59844dfa270c8eed2" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } } peers { permanent_uuid: "b78e673c552a475da13283e4193637c7" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } } }
I20260807 05:13:47.921489 32506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:47.924162 32241 master_runner.cc:263] No leader master found from master 24d28c4a7ed5489eb4065363c4218411
I20260807 05:13:47.924234 32241 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 05:13:47.924559 32517 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:13:47.924625 32517 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:13:47.933413 32526 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:13:47.940124 32526 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:13:47.957219 32316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.917283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48618 (local address 127.8.16.126:36893)
0807 05:13:47.917404 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:47.917410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:47.929832 (+ 12422us) server_negotiation.cc:407] Connection header received
0807 05:13:47.930041 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:47.930044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:47.930097 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:47.930151 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:47.933821 (+ 3670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.934630 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.940320 (+ 5690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.940639 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.940736 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.950380 (+ 9644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:47.950384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:47.950387 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:47.950416 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:47.951983 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.952481 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.952485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.952487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.952575 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.952865 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.952869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.952871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.953088 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:47.953196 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:47.956900 (+ 3704us) server_negotiation.cc:299] Negotiation successful
0807 05:13:47.957047 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 05:13:47.961786 32396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.918918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41840 (local address 127.8.16.125:35891)
0807 05:13:47.919034 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:47.919039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:47.931409 (+ 12370us) server_negotiation.cc:407] Connection header received
0807 05:13:47.933560 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:47.933564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:47.933620 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:47.933685 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:47.937555 (+ 3870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.938345 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.946165 (+ 7820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.946511 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.946591 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.954722 (+ 8131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:47.954727 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:47.954730 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:47.954764 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:47.956343 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.960301 (+ 3958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.960305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.960307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.960415 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.960743 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.960746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.960748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.960988 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:47.961104 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:47.961488 (+ 384us) server_negotiation.cc:299] Negotiation successful
0807 05:13:47.961631 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 05:13:47.963780 32441 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b78e673c552a475da13283e4193637c7 [term 1 FOLLOWER]: Refusing update from remote peer 24d28c4a7ed5489eb4065363c4218411: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:13:47.964633 32506 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b78e673c552a475da13283e4193637c7" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:13:47.967830 32474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:47.922717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60372 (local address 127.8.16.124:45081)
0807 05:13:47.922826 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:47.922830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:47.938680 (+ 15850us) server_negotiation.cc:407] Connection header received
0807 05:13:47.941589 (+ 2909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:47.941593 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:47.941654 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:47.941721 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:47.947260 (+ 5539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.948066 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.953527 (+ 5461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:47.953969 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:47.954048 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:47.963884 (+ 9836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:13:47.963888 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:13:47.963891 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:13:47.963923 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:13:47.965886 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.966425 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.966428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.966430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.966523 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:13:47.966837 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:13:47.966840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:13:47.966842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:13:47.967078 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:13:47.967192 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:47.967536 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 05:13:47.967678 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 05:13:47.969485 32477 sys_catalog.cc:455] T 00000000000000000000000000000000 P b78e673c552a475da13283e4193637c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24d28c4a7ed5489eb4065363c4218411. Latest consensus state: current_term: 1 leader_uuid: "24d28c4a7ed5489eb4065363c4218411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d28c4a7ed5489eb4065363c4218411" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36893 } } peers { permanent_uuid: "7adff94de91146b59844dfa270c8eed2" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } } peers { permanent_uuid: "b78e673c552a475da13283e4193637c7" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } } }
I20260807 05:13:47.969620 32477 sys_catalog.cc:458] T 00000000000000000000000000000000 P b78e673c552a475da13283e4193637c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:13:47.971254 32362 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7adff94de91146b59844dfa270c8eed2 [term 1 FOLLOWER]: Refusing update from remote peer 24d28c4a7ed5489eb4065363c4218411: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:13:47.971966 32524 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7adff94de91146b59844dfa270c8eed2" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:13:47.976317 32478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7adff94de91146b59844dfa270c8eed2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24d28c4a7ed5489eb4065363c4218411. Latest consensus state: current_term: 1 leader_uuid: "24d28c4a7ed5489eb4065363c4218411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d28c4a7ed5489eb4065363c4218411" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36893 } } peers { permanent_uuid: "7adff94de91146b59844dfa270c8eed2" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } } peers { permanent_uuid: "b78e673c552a475da13283e4193637c7" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } } }
I20260807 05:13:47.976455 32478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7adff94de91146b59844dfa270c8eed2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:13:47.981530 32529 mvcc.cc:205] Tried to move back new op lower bound from 7315782156091273216 to 7315782155961966592. Current Snapshot: MvccSnapshot[applied={T|T < 7315782156091273216}]
I20260807 05:13:47.982838 32530 mvcc.cc:205] Tried to move back new op lower bound from 7315782156091273216 to 7315782155961966592. Current Snapshot: MvccSnapshot[applied={T|T < 7315782156091273216}]
I20260807 05:13:47.983009 32477 sys_catalog.cc:455] T 00000000000000000000000000000000 P b78e673c552a475da13283e4193637c7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "24d28c4a7ed5489eb4065363c4218411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d28c4a7ed5489eb4065363c4218411" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36893 } } peers { permanent_uuid: "7adff94de91146b59844dfa270c8eed2" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } } peers { permanent_uuid: "b78e673c552a475da13283e4193637c7" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } } }
I20260807 05:13:47.983129 32477 sys_catalog.cc:458] T 00000000000000000000000000000000 P b78e673c552a475da13283e4193637c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:13:47.983688 32524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "24d28c4a7ed5489eb4065363c4218411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d28c4a7ed5489eb4065363c4218411" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36893 } } peers { permanent_uuid: "7adff94de91146b59844dfa270c8eed2" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } } peers { permanent_uuid: "b78e673c552a475da13283e4193637c7" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } } }
I20260807 05:13:47.983810 32524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:47.986297 32506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "24d28c4a7ed5489eb4065363c4218411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d28c4a7ed5489eb4065363c4218411" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36893 } } peers { permanent_uuid: "7adff94de91146b59844dfa270c8eed2" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } } peers { permanent_uuid: "b78e673c552a475da13283e4193637c7" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } } }
I20260807 05:13:47.986418 32506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411 [sys.catalog]: This master's current role is: LEADER
I20260807 05:13:47.987511 32532 mvcc.cc:205] Tried to move back new op lower bound from 7315782156091273216 to 7315782155961966592. Current Snapshot: MvccSnapshot[applied={T|T < 7315782156091273216}]
I20260807 05:13:47.993853 32478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7adff94de91146b59844dfa270c8eed2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "24d28c4a7ed5489eb4065363c4218411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d28c4a7ed5489eb4065363c4218411" member_type: VOTER last_known_addr { host: "127.8.16.126" port: 36893 } } peers { permanent_uuid: "7adff94de91146b59844dfa270c8eed2" member_type: VOTER last_known_addr { host: "127.8.16.125" port: 35891 } } peers { permanent_uuid: "b78e673c552a475da13283e4193637c7" member_type: VOTER last_known_addr { host: "127.8.16.124" port: 45081 } } }
I20260807 05:13:47.994006 32478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7adff94de91146b59844dfa270c8eed2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:13:47.998713 32526 catalog_manager.cc:1347] Generated new cluster ID: d8e473c32cbc4e42ad00c360e210e342
I20260807 05:13:47.998797 32526 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:13:48.033792 32526 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:13:48.037509 32526 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:13:48.057883 32526 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24d28c4a7ed5489eb4065363c4218411: Generated new TSK 0
I20260807 05:13:48.058714 32526 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 05:13:48.091578 32316 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:48.088426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48626 (local address 127.8.16.126:36893)
0807 05:13:48.088557 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:48.088562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:48.090435 (+ 1873us) server_negotiation.cc:407] Connection header received
0807 05:13:48.090647 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:48.090650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:48.090703 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:48.090757 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:48.091437 (+ 680us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48626: BlockingRecv error: recv got EOF from 127.0.0.1:48626 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 05:13:48.093142 32396 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:48.089191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41844 (local address 127.8.16.125:35891)
0807 05:13:48.089306 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:48.089310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:48.092135 (+ 2825us) server_negotiation.cc:407] Connection header received
0807 05:13:48.092338 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:48.092341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:48.092395 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:48.092453 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:48.093020 (+ 567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41844: BlockingRecv error: recv got EOF from 127.0.0.1:41844 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260807 05:13:48.094820 32474 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:48.089871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60388 (local address 127.8.16.124:45081)
0807 05:13:48.089978 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:48.089982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:48.093740 (+ 3758us) server_negotiation.cc:407] Connection header received
0807 05:13:48.093913 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:48.093916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:48.093969 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:48.094035 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:48.094677 (+ 642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60388: BlockingRecv error: recv got EOF from 127.0.0.1:60388 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260807 05:13:48.102916 32396 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:48.100604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41860 (local address 127.8.16.125:35891)
0807 05:13:48.100725 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:48.100730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:48.101800 (+ 1070us) server_negotiation.cc:407] Connection header received
0807 05:13:48.102042 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:48.102045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:48.102120 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:48.102171 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:48.102790 (+ 619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41860: BlockingRecv error: recv got EOF from 127.0.0.1:41860 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 05:13:48.111043 32396 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:48.108987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41864 (local address 127.8.16.125:35891)
0807 05:13:48.109113 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:48.109118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:48.109782 (+ 664us) server_negotiation.cc:407] Connection header received
0807 05:13:48.110074 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:48.110077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:48.110136 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:48.110191 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:48.110892 (+ 701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41864: BlockingRecv error: recv got EOF from 127.0.0.1:41864 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 05:13:48.116392 32396 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:48.115248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41874 (local address 127.8.16.125:35891)
0807 05:13:48.115349 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:48.115353 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:48.115609 (+ 256us) server_negotiation.cc:407] Connection header received
0807 05:13:48.115717 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:48.115720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:48.115775 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:48.115864 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:48.116279 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41874: BlockingRecv error: recv got EOF from 127.0.0.1:41874 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260807 05:13:48.123147 32396 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:48.122175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41886 (local address 127.8.16.125:35891)
0807 05:13:48.122309 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:48.122312 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:48.122402 (+ 90us) server_negotiation.cc:407] Connection header received
0807 05:13:48.122518 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:48.122520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:48.122560 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:48.122623 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:48.123043 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41886: BlockingRecv error: recv got EOF from 127.0.0.1:41886 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260807 05:13:48.134363 32396 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:48.133233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41898 (local address 127.8.16.125:35891)
0807 05:13:48.133380 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:48.133383 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:48.133516 (+ 133us) server_negotiation.cc:407] Connection header received
0807 05:13:48.133654 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:48.133657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:48.133702 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:48.133777 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:48.134248 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41898: BlockingRecv error: recv got EOF from 127.0.0.1:41898 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 05:13:48.153424 32396 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:48.152274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41908 (local address 127.8.16.125:35891)
0807 05:13:48.152430 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:48.152435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:48.152521 (+ 86us) server_negotiation.cc:407] Connection header received
0807 05:13:48.152689 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:48.152692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:48.152744 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:48.152815 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:48.153300 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41908: BlockingRecv error: recv got EOF from 127.0.0.1:41908 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 05:13:48.188602 32396 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:48.187470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41924 (local address 127.8.16.125:35891)
0807 05:13:48.187621 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:48.187626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:48.187734 (+ 108us) server_negotiation.cc:407] Connection header received
0807 05:13:48.187883 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:48.187886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:48.187937 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:48.188008 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:48.188482 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41924: BlockingRecv error: recv got EOF from 127.0.0.1:41924 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 05:13:48.255684 32396 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:48.254680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41928 (local address 127.8.16.125:35891)
0807 05:13:48.254855 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:48.254859 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:48.254920 (+ 61us) server_negotiation.cc:407] Connection header received
0807 05:13:48.255047 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:48.255049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:48.255099 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:48.255168 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:48.255560 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41928: BlockingRecv error: recv got EOF from 127.0.0.1:41928 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260807 05:13:48.386955 32396 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:48.385877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41944 (local address 127.8.16.125:35891)
0807 05:13:48.385998 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:48.386002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:48.386181 (+ 179us) server_negotiation.cc:407] Connection header received
0807 05:13:48.386315 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:48.386318 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:48.386367 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:48.386441 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:48.386847 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41944: BlockingRecv error: recv got EOF from 127.0.0.1:41944 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260807 05:13:48.646118 32396 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:13:48.645088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41952 (local address 127.8.16.125:35891)
0807 05:13:48.645256 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:48.645260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:48.645311 (+ 51us) server_negotiation.cc:407] Connection header received
0807 05:13:48.645474 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:48.645477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:48.645525 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:48.645586 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:13:48.645979 (+ 393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41952: BlockingRecv error: recv got EOF from 127.0.0.1:41952 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 05:13:48.904631 32490 catalog_manager.cc:1259] Loaded cluster ID: d8e473c32cbc4e42ad00c360e210e342
I20260807 05:13:48.904724 32490 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b78e673c552a475da13283e4193637c7: loading cluster ID for follower catalog manager: success
I20260807 05:13:48.906617 32490 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b78e673c552a475da13283e4193637c7: acquiring CA information for follower catalog manager: success
I20260807 05:13:48.907840 32490 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b78e673c552a475da13283e4193637c7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 05:13:48.913394 32502 catalog_manager.cc:1259] Loaded cluster ID: d8e473c32cbc4e42ad00c360e210e342
I20260807 05:13:48.913481 32502 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7adff94de91146b59844dfa270c8eed2: loading cluster ID for follower catalog manager: success
I20260807 05:13:48.915117 32502 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7adff94de91146b59844dfa270c8eed2: acquiring CA information for follower catalog manager: success
I20260807 05:13:48.916185 32502 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7adff94de91146b59844dfa270c8eed2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 05:13:49.164176 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:49.160156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41960 (local address 127.8.16.125:35891)
0807 05:13:49.160408 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:49.160413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:49.160431 (+ 18us) server_negotiation.cc:407] Connection header received
0807 05:13:49.160593 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:49.160596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:49.160734 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:49.160840 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:13:49.161099 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:49.161685 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:49.162590 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:49.162868 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:49.162962 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:49.163279 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:13:49.163779 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:13:49.163895 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:49.163960 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 05:13:49.164020 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":81,"threads_started":1}
I20260807 05:13:49.170104 32608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:49.166977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60400 (local address 127.8.16.124:45081)
0807 05:13:49.167223 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:49.167226 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:49.167240 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:13:49.167313 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:49.167315 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:49.167420 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:49.167512 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:13:49.167668 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:49.168168 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:49.168893 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:49.169099 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:49.169163 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:49.169401 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:13:49.169833 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:13:49.169917 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:49.169942 (+ 25us) server_negotiation.cc:299] Negotiation successful
0807 05:13:49.169986 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":81,"threads_started":1}
I20260807 05:13:49.177016 32613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:49.172694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48638 (local address 127.8.16.126:36893)
0807 05:13:49.172929 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:49.172934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:49.173182 (+ 248us) server_negotiation.cc:407] Connection header received
0807 05:13:49.173326 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:49.173329 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:49.173507 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:49.173607 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:13:49.174001 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:49.174780 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:49.175627 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:49.175854 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:49.175936 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:49.176293 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:13:49.176626 (+ 333us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:13:49.176708 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:49.176801 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 05:13:49.176862 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":69,"threads_started":1}
I20260807 05:13:49.177343 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:49.173247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41964 (local address 127.8.16.125:35891)
0807 05:13:49.173353 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:49.173356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:49.173794 (+ 438us) server_negotiation.cc:407] Connection header received
0807 05:13:49.173879 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:49.173882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:49.173936 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:49.174000 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:13:49.174205 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:49.174684 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:49.176194 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:49.176483 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:49.176584 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:49.176857 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:13:49.176939 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:13:49.176988 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:49.177163 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 05:13:49.177226 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 05:13:49.177747 32608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:13:49.172828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60408 (local address 127.8.16.124:45081)
0807 05:13:49.172927 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 05:13:49.172930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:13:49.173280 (+ 350us) server_negotiation.cc:407] Connection header received
0807 05:13:49.173472 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:13:49.173475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:13:49.173523 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:13:49.173592 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:13:49.173955 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:49.174727 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:49.176652 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:13:49.176962 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:13:49.177053 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:13:49.177353 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:13:49.177415 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:13:49.177497 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 05:13:49.177581 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 05:13:49.177629 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 05:13:49.180413 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 32241
I20260807 05:13:49.187294 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 32319
I20260807 05:13:49.193226 8257 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskg_K4zd/build/debug/bin/kudu with pid 32397
2026-08-07T05:13:49Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3079 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6126 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (275115 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST